Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2400,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2400,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.668545
Epoch 1
Validation auc = 0.681221
Epoch 2
Validation auc = 0.691549
Epoch 3
Validation auc = 0.718310
Epoch 4
Loss = 6.1967e-01, PNorm = 100.5485, GNorm = 2.9180, lr_0 = 6.5000e-04
Validation auc = 0.616432
Epoch 5
Validation auc = 0.671362
Epoch 6
Validation auc = 0.603756
Epoch 7
Validation auc = 0.611737
Epoch 8
Validation auc = 0.688263
Epoch 9
Loss = 4.8144e-01, PNorm = 100.9090, GNorm = 6.3450, lr_0 = 9.9743e-04
Validation auc = 0.696244
Epoch 10
Validation auc = 0.713615
Epoch 11
Validation auc = 0.680751
Epoch 12
Validation auc = 0.696714
Epoch 13
Validation auc = 0.691080
Epoch 14
Loss = 3.6447e-01, PNorm = 101.2727, GNorm = 1.3392, lr_0 = 9.8890e-04
Validation auc = 0.695775
Epoch 15
Validation auc = 0.707042
Epoch 16
Validation auc = 0.692488
Epoch 17
Validation auc = 0.683568
Epoch 18
Validation auc = 0.635211
Epoch 19
Loss = 3.3827e-01, PNorm = 101.5788, GNorm = 2.5860, lr_0 = 9.8045e-04
Validation auc = 0.649765
Epoch 20
Validation auc = 0.640845
Epoch 21
Validation auc = 0.653521
Epoch 22
Validation auc = 0.688263
Epoch 23
Validation auc = 0.723005
Epoch 24
Loss = 2.3762e-01, PNorm = 101.8752, GNorm = 3.0602, lr_0 = 9.7207e-04
Validation auc = 0.731455
Epoch 25
Validation auc = 0.716432
Epoch 26
Validation auc = 0.707981
Epoch 27
Validation auc = 0.712676
Epoch 28
Validation auc = 0.732864
Epoch 29
Loss = 2.0179e-01, PNorm = 102.1680, GNorm = 2.5326, lr_0 = 9.6376e-04
Validation auc = 0.733803
Epoch 30
Validation auc = 0.727230
Epoch 31
Validation auc = 0.714085
Epoch 32
Validation auc = 0.699531
Epoch 33
Validation auc = 0.685915
Epoch 34
Loss = 2.2647e-01, PNorm = 102.5167, GNorm = 8.4543, lr_0 = 9.5552e-04
Validation auc = 0.690141
Epoch 35
Validation auc = 0.691080
Epoch 36
Validation auc = 0.700469
Epoch 37
Validation auc = 0.707981
Epoch 38
Validation auc = 0.697183
Epoch 39
Loss = 2.6637e-01, PNorm = 102.8772, GNorm = 8.5114, lr_0 = 9.4735e-04
Validation auc = 0.678404
Epoch 40
Validation auc = 0.688263
Epoch 41
Validation auc = 0.682160
Epoch 42
Validation auc = 0.669484
Epoch 43
Validation auc = 0.674648
Epoch 44
Loss = 1.7063e-01, PNorm = 103.2716, GNorm = 2.1057, lr_0 = 9.3925e-04
Validation auc = 0.675587
Epoch 45
Validation auc = 0.682629
Epoch 46
Validation auc = 0.681690
Epoch 47
Validation auc = 0.691549
Epoch 48
Validation auc = 0.691080
Epoch 49
Loss = 1.3222e-01, PNorm = 103.6348, GNorm = 1.2827, lr_0 = 9.3122e-04
Validation auc = 0.702347
Epoch 50
Validation auc = 0.722066
Epoch 51
Validation auc = 0.716901
Epoch 52
Validation auc = 0.705634
Epoch 53
Validation auc = 0.695305
Epoch 54
Loss = 7.0185e-02, PNorm = 103.9946, GNorm = 1.1786, lr_0 = 9.2326e-04
Validation auc = 0.685915
Epoch 55
Validation auc = 0.653521
Epoch 56
Validation auc = 0.682160
Epoch 57
Validation auc = 0.700939
Epoch 58
Validation auc = 0.711268
Epoch 59
Loss = 1.0060e-01, PNorm = 104.3438, GNorm = 1.1682, lr_0 = 9.1537e-04
Validation auc = 0.710329
Epoch 60
Validation auc = 0.697183
Epoch 61
Validation auc = 0.705164
Epoch 62
Validation auc = 0.700469
Epoch 63
Validation auc = 0.695775
Epoch 64
Loss = 2.2598e-02, PNorm = 104.7576, GNorm = 1.0565, lr_0 = 9.0754e-04
Validation auc = 0.677465
Epoch 65
Validation auc = 0.700000
Epoch 66
Validation auc = 0.698122
Epoch 67
Validation auc = 0.679343
Epoch 68
Validation auc = 0.709390
Epoch 69
Loss = 1.2925e-01, PNorm = 105.1640, GNorm = 6.6538, lr_0 = 8.9978e-04
Validation auc = 0.723474
Epoch 70
Validation auc = 0.731925
Epoch 71
Validation auc = 0.734742
Epoch 72
Validation auc = 0.741784
Epoch 73
Validation auc = 0.728169
Epoch 74
Loss = 3.7331e-02, PNorm = 105.5916, GNorm = 0.4199, lr_0 = 8.9209e-04
Validation auc = 0.708451
Epoch 75
Validation auc = 0.690141
Epoch 76
Validation auc = 0.694836
Epoch 77
Validation auc = 0.700939
Epoch 78
Validation auc = 0.708920
Epoch 79
Loss = 7.0748e-02, PNorm = 105.9121, GNorm = 3.2048, lr_0 = 8.8447e-04
Validation auc = 0.724413
Epoch 80
Validation auc = 0.699531
Epoch 81
Validation auc = 0.683568
Epoch 82
Validation auc = 0.696244
Epoch 83
Validation auc = 0.694366
Epoch 84
Loss = 1.2775e-01, PNorm = 106.2205, GNorm = 3.6468, lr_0 = 8.7691e-04
Validation auc = 0.710329
Epoch 85
Validation auc = 0.719249
Epoch 86
Validation auc = 0.711737
Epoch 87
Validation auc = 0.706103
Epoch 88
Validation auc = 0.703286
Epoch 89
Loss = 1.3968e-01, PNorm = 106.5330, GNorm = 1.9672, lr_0 = 8.6941e-04
Validation auc = 0.694836
Epoch 90
Validation auc = 0.707512
Epoch 91
Validation auc = 0.705164
Epoch 92
Validation auc = 0.699061
Epoch 93
Validation auc = 0.713146
Epoch 94
Loss = 5.0018e-02, PNorm = 106.8267, GNorm = 2.2628, lr_0 = 8.6198e-04
Validation auc = 0.727230
Epoch 95
Validation auc = 0.720188
Epoch 96
Validation auc = 0.713146
Epoch 97
Validation auc = 0.693427
Epoch 98
Validation auc = 0.684507
Epoch 99
Loss = 2.1711e-02, PNorm = 107.0684, GNorm = 0.3457, lr_0 = 8.5461e-04
Validation auc = 0.670892
Epoch 100
Validation auc = 0.678873
Epoch 101
Validation auc = 0.676995
Epoch 102
Validation auc = 0.690610
Epoch 103
Validation auc = 0.705164
Epoch 104
Loss = 2.4669e-02, PNorm = 107.2571, GNorm = 2.1540, lr_0 = 8.4730e-04
Validation auc = 0.715493
Epoch 105
Validation auc = 0.724883
Epoch 106
Validation auc = 0.730047
Epoch 107
Validation auc = 0.735211
Epoch 108
Validation auc = 0.733333
Epoch 109
Loss = 2.4847e-02, PNorm = 107.4052, GNorm = 1.2743, lr_0 = 8.4006e-04
Validation auc = 0.734272
Epoch 110
Validation auc = 0.735211
Epoch 111
Validation auc = 0.738498
Epoch 112
Validation auc = 0.743192
Epoch 113
Validation auc = 0.746479
Epoch 114
Loss = 5.4746e-02, PNorm = 107.5525, GNorm = 1.7941, lr_0 = 8.3288e-04
Validation auc = 0.737559
Epoch 115
Validation auc = 0.734742
Epoch 116
Validation auc = 0.735681
Epoch 117
Validation auc = 0.731455
Epoch 118
Validation auc = 0.730516
Epoch 119
Loss = 3.5688e-02, PNorm = 107.7365, GNorm = 0.5954, lr_0 = 8.2576e-04
Validation auc = 0.731455
Epoch 120
Validation auc = 0.704695
Epoch 121
Validation auc = 0.693427
Epoch 122
Validation auc = 0.695305
Epoch 123
Validation auc = 0.679343
Epoch 124
Loss = 4.2260e-01, PNorm = 108.0016, GNorm = 7.8325, lr_0 = 8.1870e-04
Validation auc = 0.641315
Epoch 125
Validation auc = 0.654460
Epoch 126
Validation auc = 0.656338
Epoch 127
Validation auc = 0.670423
Epoch 128
Validation auc = 0.670423
Epoch 129
Loss = 3.2640e-01, PNorm = 108.5840, GNorm = 6.9452, lr_0 = 8.1170e-04
Validation auc = 0.681221
Epoch 130
Validation auc = 0.674648
Epoch 131
Validation auc = 0.689202
Epoch 132
Validation auc = 0.703286
Epoch 133
Validation auc = 0.709859
Epoch 134
Loss = 4.8445e-02, PNorm = 109.0665, GNorm = 0.8024, lr_0 = 8.0476e-04
Validation auc = 0.708451
Epoch 135
Validation auc = 0.706103
Epoch 136
Validation auc = 0.698592
Epoch 137
Validation auc = 0.696244
Epoch 138
Validation auc = 0.697653
Epoch 139
Loss = 4.2758e-02, PNorm = 109.3857, GNorm = 1.2983, lr_0 = 7.9788e-04
Validation auc = 0.688732
Epoch 140
Validation auc = 0.685446
Epoch 141
Validation auc = 0.680282
Epoch 142
Validation auc = 0.669484
Epoch 143
Validation auc = 0.668075
Epoch 144
Loss = 3.1129e-02, PNorm = 109.6298, GNorm = 0.5307, lr_0 = 7.9106e-04
Validation auc = 0.669953
Epoch 145
Validation auc = 0.690141
Epoch 146
Validation auc = 0.704695
Epoch 147
Validation auc = 0.705164
Epoch 148
Validation auc = 0.693897
Epoch 149
Loss = 2.5823e-02, PNorm = 109.8053, GNorm = 1.6914, lr_0 = 7.8430e-04
Validation auc = 0.687324
Epoch 150
Validation auc = 0.697653
Epoch 151
Validation auc = 0.708451
Epoch 152
Validation auc = 0.710329
Epoch 153
Validation auc = 0.707512
Epoch 154
Loss = 1.1948e-02, PNorm = 109.9504, GNorm = 0.4816, lr_0 = 7.7759e-04
Validation auc = 0.715962
Epoch 155
Validation auc = 0.717840
Epoch 156
Validation auc = 0.722535
Epoch 157
Validation auc = 0.723474
Epoch 158
Validation auc = 0.715493
Epoch 159
Loss = 1.1562e-02, PNorm = 110.0738, GNorm = 0.8398, lr_0 = 7.7095e-04
Validation auc = 0.708451
Epoch 160
Validation auc = 0.700939
Epoch 161
Validation auc = 0.696714
Epoch 162
Validation auc = 0.692019
Epoch 163
Validation auc = 0.689202
Epoch 164
Loss = 4.5195e-02, PNorm = 110.1713, GNorm = 1.1346, lr_0 = 7.6436e-04
Validation auc = 0.710329
Epoch 165
Validation auc = 0.705634
Epoch 166
Validation auc = 0.695775
Epoch 167
Validation auc = 0.692019
Epoch 168
Validation auc = 0.686385
Epoch 169
Loss = 2.9234e-03, PNorm = 110.2783, GNorm = 0.6727, lr_0 = 7.5782e-04
Validation auc = 0.690610
Epoch 170
Validation auc = 0.688263
Epoch 171
Validation auc = 0.685446
Epoch 172
Validation auc = 0.684038
Epoch 173
Validation auc = 0.680282
Epoch 174
Loss = 4.9723e-03, PNorm = 110.3919, GNorm = 0.0605, lr_0 = 7.5134e-04
Validation auc = 0.681690
Epoch 175
Validation auc = 0.688732
Epoch 176
Validation auc = 0.691080
Epoch 177
Validation auc = 0.692488
Epoch 178
Validation auc = 0.698592
Epoch 179
Loss = 5.6792e-02, PNorm = 110.5001, GNorm = 3.2894, lr_0 = 7.4492e-04
Validation auc = 0.698122
Epoch 180
Validation auc = 0.694836
Epoch 181
Validation auc = 0.701878
Epoch 182
Validation auc = 0.710329
Epoch 183
Validation auc = 0.709859
Epoch 184
Loss = 2.0133e-02, PNorm = 110.6363, GNorm = 2.1750, lr_0 = 7.3855e-04
Validation auc = 0.707512
Epoch 185
Validation auc = 0.704225
Epoch 186
Validation auc = 0.696714
Epoch 187
Validation auc = 0.699531
Epoch 188
Validation auc = 0.698122
Epoch 189
Loss = 2.6422e-02, PNorm = 110.7880, GNorm = 0.6598, lr_0 = 7.3224e-04
Validation auc = 0.695775
Epoch 190
Validation auc = 0.694836
Epoch 191
Validation auc = 0.705634
Epoch 192
Validation auc = 0.712676
Epoch 193
Validation auc = 0.709859
Epoch 194
Loss = 5.6734e-03, PNorm = 110.9156, GNorm = 0.3942, lr_0 = 7.2598e-04
Validation auc = 0.712676
Epoch 195
Validation auc = 0.723474
Epoch 196
Validation auc = 0.732864
Epoch 197
Validation auc = 0.730986
Epoch 198
Validation auc = 0.717840
Epoch 199
Loss = 4.2765e-03, PNorm = 111.0411, GNorm = 0.8660, lr_0 = 7.1977e-04
Validation auc = 0.708451
Epoch 200
Validation auc = 0.698592
Epoch 201
Validation auc = 0.691080
Epoch 202
Validation auc = 0.688732
Epoch 203
Validation auc = 0.691080
Epoch 204
Loss = 2.3707e-03, PNorm = 111.1600, GNorm = 0.1040, lr_0 = 7.1362e-04
Validation auc = 0.693897
Epoch 205
Validation auc = 0.692488
Epoch 206
Validation auc = 0.697653
Epoch 207
Validation auc = 0.702347
Epoch 208
Validation auc = 0.706573
Epoch 209
Loss = 3.0047e-03, PNorm = 111.2638, GNorm = 0.0234, lr_0 = 7.0752e-04
Validation auc = 0.712207
Epoch 210
Validation auc = 0.713146
Epoch 211
Validation auc = 0.715962
Epoch 212
Validation auc = 0.706103
Epoch 213
Validation auc = 0.698122
Epoch 214
Loss = 1.0735e-03, PNorm = 111.3573, GNorm = 0.1532, lr_0 = 7.0147e-04
Validation auc = 0.698122
Epoch 215
Validation auc = 0.699061
Epoch 216
Validation auc = 0.699531
Epoch 217
Validation auc = 0.697653
Epoch 218
Validation auc = 0.697183
Epoch 219
Loss = 2.9587e-04, PNorm = 111.4289, GNorm = 0.0204, lr_0 = 6.9548e-04
Validation auc = 0.697183
Epoch 220
Validation auc = 0.696714
Epoch 221
Validation auc = 0.695775
Epoch 222
Validation auc = 0.700469
Epoch 223
Validation auc = 0.702347
Epoch 224
Loss = 1.9457e-03, PNorm = 111.4693, GNorm = 0.1199, lr_0 = 6.8953e-04
Validation auc = 0.702347
Epoch 225
Validation auc = 0.699531
Epoch 226
Validation auc = 0.698122
Epoch 227
Validation auc = 0.696244
Epoch 228
Validation auc = 0.696244
Epoch 229
Loss = 4.3738e-04, PNorm = 111.4963, GNorm = 0.0186, lr_0 = 6.8364e-04
Validation auc = 0.694836
Epoch 230
Validation auc = 0.692958
Epoch 231
Validation auc = 0.692488
Epoch 232
Validation auc = 0.692488
Epoch 233
Validation auc = 0.692488
Epoch 234
Loss = 2.5422e-04, PNorm = 111.5131, GNorm = 0.0155, lr_0 = 6.7779e-04
Validation auc = 0.692019
Epoch 235
Validation auc = 0.692019
Epoch 236
Validation auc = 0.691549
Epoch 237
Validation auc = 0.692019
Epoch 238
Validation auc = 0.691549
Epoch 239
Loss = 3.5787e-04, PNorm = 111.5225, GNorm = 0.0954, lr_0 = 6.7200e-04
Validation auc = 0.691549
Epoch 240
Validation auc = 0.691549
Epoch 241
Validation auc = 0.691549
Epoch 242
Validation auc = 0.692019
Epoch 243
Validation auc = 0.690610
Epoch 244
Loss = 1.1949e-04, PNorm = 111.5295, GNorm = 0.0020, lr_0 = 6.6625e-04
Validation auc = 0.692019
Epoch 245
Validation auc = 0.692019
Epoch 246
Validation auc = 0.692019
Epoch 247
Validation auc = 0.692958
Epoch 248
Validation auc = 0.693897
Epoch 249
Loss = 3.4085e-04, PNorm = 111.5378, GNorm = 0.0597, lr_0 = 6.6056e-04
Validation auc = 0.692019
Epoch 250
Validation auc = 0.693427
Epoch 251
Validation auc = 0.695305
Epoch 252
Validation auc = 0.696244
Epoch 253
Validation auc = 0.697183
Epoch 254
Loss = 4.4169e-04, PNorm = 111.5494, GNorm = 0.0180, lr_0 = 6.5491e-04
Validation auc = 0.699061
Epoch 255
Validation auc = 0.698592
Epoch 256
Validation auc = 0.696244
Epoch 257
Validation auc = 0.693427
Epoch 258
Validation auc = 0.691080
Epoch 259
Loss = 2.1936e-04, PNorm = 111.5604, GNorm = 0.0025, lr_0 = 6.4931e-04
Validation auc = 0.691549
Epoch 260
Validation auc = 0.689671
Epoch 261
Validation auc = 0.689202
Epoch 262
Validation auc = 0.687793
Epoch 263
Validation auc = 0.686854
Epoch 264
Loss = 1.9638e-03, PNorm = 111.5698, GNorm = 0.5478, lr_0 = 6.4376e-04
Validation auc = 0.686854
Epoch 265
Validation auc = 0.687324
Epoch 266
Validation auc = 0.691080
Epoch 267
Validation auc = 0.692019
Epoch 268
Validation auc = 0.692019
Epoch 269
Loss = 1.9470e-04, PNorm = 111.5908, GNorm = 0.0204, lr_0 = 6.3826e-04
Validation auc = 0.690141
Epoch 270
Validation auc = 0.691080
Epoch 271
Validation auc = 0.689671
Epoch 272
Validation auc = 0.688732
Epoch 273
Validation auc = 0.686854
Epoch 274
Loss = 4.5879e-05, PNorm = 111.6089, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation auc = 0.685915
Epoch 275
Validation auc = 0.685915
Epoch 276
Validation auc = 0.684977
Epoch 277
Validation auc = 0.685446
Epoch 278
Validation auc = 0.684507
Epoch 279
Loss = 2.7363e-03, PNorm = 111.6186, GNorm = 0.8493, lr_0 = 6.2739e-04
Validation auc = 0.686385
Epoch 280
Validation auc = 0.694366
Epoch 281
Validation auc = 0.701408
Epoch 282
Validation auc = 0.704225
Epoch 283
Validation auc = 0.705164
Epoch 284
Loss = 1.1703e-04, PNorm = 111.6307, GNorm = 0.0115, lr_0 = 6.2203e-04
Validation auc = 0.707512
Epoch 285
Validation auc = 0.706573
Epoch 286
Validation auc = 0.707512
Epoch 287
Validation auc = 0.708451
Epoch 288
Validation auc = 0.709859
Epoch 289
Loss = 4.9242e-05, PNorm = 111.6416, GNorm = 0.0063, lr_0 = 6.1671e-04
Validation auc = 0.708920
Epoch 290
Validation auc = 0.709390
Epoch 291
Validation auc = 0.708920
Epoch 292
Validation auc = 0.707042
Epoch 293
Validation auc = 0.707042
Epoch 294
Loss = 4.1414e-05, PNorm = 111.6477, GNorm = 0.0032, lr_0 = 6.1144e-04
Validation auc = 0.706573
Epoch 295
Validation auc = 0.706103
Epoch 296
Validation auc = 0.705164
Epoch 297
Validation auc = 0.705634
Epoch 298
Validation auc = 0.705634
Epoch 299
Loss = 2.7608e-04, PNorm = 111.6514, GNorm = 0.0719, lr_0 = 6.0621e-04
Validation auc = 0.705634
Model 0 best validation auc = 0.746479 on epoch 113
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.886213
Ensemble test auc = 0.886213
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.696244
Epoch 1
Validation auc = 0.639437
Epoch 2
Validation auc = 0.638967
Epoch 3
Validation auc = 0.670423
Epoch 4
Loss = 5.8919e-01, PNorm = 100.5473, GNorm = 4.3928, lr_0 = 6.5000e-04
Validation auc = 0.573239
Epoch 5
Validation auc = 0.670423
Epoch 6
Validation auc = 0.668075
Epoch 7
Validation auc = 0.663380
Epoch 8
Validation auc = 0.630516
Epoch 9
Loss = 4.6040e-01, PNorm = 100.9141, GNorm = 2.6647, lr_0 = 9.9743e-04
Validation auc = 0.636620
Epoch 10
Validation auc = 0.671831
Epoch 11
Validation auc = 0.682160
Epoch 12
Validation auc = 0.683099
Epoch 13
Validation auc = 0.677465
Epoch 14
Loss = 4.1848e-01, PNorm = 101.2884, GNorm = 5.6892, lr_0 = 9.8890e-04
Validation auc = 0.664789
Epoch 15
Validation auc = 0.672300
Epoch 16
Validation auc = 0.666197
Epoch 17
Validation auc = 0.684038
Epoch 18
Validation auc = 0.669484
Epoch 19
Loss = 3.4289e-01, PNorm = 101.5962, GNorm = 1.4820, lr_0 = 9.8045e-04
Validation auc = 0.672770
Epoch 20
Validation auc = 0.662441
Epoch 21
Validation auc = 0.662441
Epoch 22
Validation auc = 0.664789
Epoch 23
Validation auc = 0.669953
Epoch 24
Loss = 2.8708e-01, PNorm = 101.9071, GNorm = 3.9191, lr_0 = 9.7207e-04
Validation auc = 0.667136
Epoch 25
Validation auc = 0.646479
Epoch 26
Validation auc = 0.657746
Epoch 27
Validation auc = 0.662911
Epoch 28
Validation auc = 0.662911
Epoch 29
Loss = 2.1493e-01, PNorm = 102.2162, GNorm = 1.8131, lr_0 = 9.6376e-04
Validation auc = 0.665728
Epoch 30
Validation auc = 0.671362
Epoch 31
Validation auc = 0.684038
Epoch 32
Validation auc = 0.647887
Epoch 33
Validation auc = 0.680751
Epoch 34
Loss = 1.4107e-01, PNorm = 102.5502, GNorm = 1.4835, lr_0 = 9.5552e-04
Validation auc = 0.675587
Epoch 35
Validation auc = 0.682160
Epoch 36
Validation auc = 0.698122
Epoch 37
Validation auc = 0.706103
Epoch 38
Validation auc = 0.693897
Epoch 39
Loss = 1.9366e-01, PNorm = 102.8955, GNorm = 2.8742, lr_0 = 9.4735e-04
Validation auc = 0.692488
Epoch 40
Validation auc = 0.688732
Epoch 41
Validation auc = 0.696244
Epoch 42
Validation auc = 0.700469
Epoch 43
Validation auc = 0.715962
Epoch 44
Loss = 2.7958e-01, PNorm = 103.2290, GNorm = 15.4365, lr_0 = 9.3925e-04
Validation auc = 0.741315
Epoch 45
Validation auc = 0.747418
Epoch 46
Validation auc = 0.719249
Epoch 47
Validation auc = 0.698122
Epoch 48
Validation auc = 0.644601
Epoch 49
Loss = 1.7753e-01, PNorm = 103.6667, GNorm = 3.7000, lr_0 = 9.3122e-04
Validation auc = 0.654930
Epoch 50
Validation auc = 0.670423
Epoch 51
Validation auc = 0.687793
Epoch 52
Validation auc = 0.701878
Epoch 53
Validation auc = 0.698122
Epoch 54
Loss = 1.8789e-01, PNorm = 104.0337, GNorm = 2.0005, lr_0 = 9.2326e-04
Validation auc = 0.699061
Epoch 55
Validation auc = 0.692488
Epoch 56
Validation auc = 0.681221
Epoch 57
Validation auc = 0.685446
Epoch 58
Validation auc = 0.689202
Epoch 59
Loss = 6.3917e-02, PNorm = 104.3881, GNorm = 2.0778, lr_0 = 9.1537e-04
Validation auc = 0.675587
Epoch 60
Validation auc = 0.671362
Epoch 61
Validation auc = 0.668545
Epoch 62
Validation auc = 0.670892
Epoch 63
Validation auc = 0.677465
Epoch 64
Loss = 8.5712e-02, PNorm = 104.6872, GNorm = 1.6771, lr_0 = 9.0754e-04
Validation auc = 0.696714
Epoch 65
Validation auc = 0.697183
Epoch 66
Validation auc = 0.701878
Epoch 67
Validation auc = 0.712207
Epoch 68
Validation auc = 0.715493
Epoch 69
Loss = 1.2973e-01, PNorm = 105.0112, GNorm = 7.3613, lr_0 = 8.9978e-04
Validation auc = 0.710329
Epoch 70
Validation auc = 0.701408
Epoch 71
Validation auc = 0.687793
Epoch 72
Validation auc = 0.687793
Epoch 73
Validation auc = 0.692488
Epoch 74
Loss = 1.1524e-01, PNorm = 105.3532, GNorm = 1.8252, lr_0 = 8.9209e-04
Validation auc = 0.697653
Epoch 75
Validation auc = 0.696244
Epoch 76
Validation auc = 0.679343
Epoch 77
Validation auc = 0.659155
Epoch 78
Validation auc = 0.687324
Epoch 79
Loss = 6.7329e-02, PNorm = 105.7971, GNorm = 2.4483, lr_0 = 8.8447e-04
Validation auc = 0.695305
Epoch 80
Validation auc = 0.682629
Epoch 81
Validation auc = 0.690610
Epoch 82
Validation auc = 0.709859
Epoch 83
Validation auc = 0.709859
Epoch 84
Loss = 1.4486e-01, PNorm = 106.2491, GNorm = 2.1246, lr_0 = 8.7691e-04
Validation auc = 0.706103
Epoch 85
Validation auc = 0.704225
Epoch 86
Validation auc = 0.707512
Epoch 87
Validation auc = 0.713615
Epoch 88
Validation auc = 0.712207
Epoch 89
Loss = 1.1371e-01, PNorm = 106.6214, GNorm = 4.6795, lr_0 = 8.6941e-04
Validation auc = 0.705164
Epoch 90
Validation auc = 0.734742
Epoch 91
Validation auc = 0.733803
Epoch 92
Validation auc = 0.742254
Epoch 93
Validation auc = 0.737089
Epoch 94
Loss = 6.6626e-02, PNorm = 106.9481, GNorm = 4.3175, lr_0 = 8.6198e-04
Validation auc = 0.728638
Epoch 95
Validation auc = 0.719718
Epoch 96
Validation auc = 0.728638
Epoch 97
Validation auc = 0.731925
Epoch 98
Validation auc = 0.735681
Epoch 99
Loss = 7.3039e-02, PNorm = 107.2857, GNorm = 2.5255, lr_0 = 8.5461e-04
Validation auc = 0.718310
Epoch 100
Validation auc = 0.715023
Epoch 101
Validation auc = 0.687793
Epoch 102
Validation auc = 0.669953
Epoch 103
Validation auc = 0.656338
Epoch 104
Loss = 1.0200e-01, PNorm = 107.6425, GNorm = 4.2292, lr_0 = 8.4730e-04
Validation auc = 0.666667
Epoch 105
Validation auc = 0.674648
Epoch 106
Validation auc = 0.693427
Epoch 107
Validation auc = 0.707981
Epoch 108
Validation auc = 0.714554
Epoch 109
Loss = 3.0924e-02, PNorm = 107.9196, GNorm = 0.5186, lr_0 = 8.4006e-04
Validation auc = 0.706103
Epoch 110
Validation auc = 0.704695
Epoch 111
Validation auc = 0.702817
Epoch 112
Validation auc = 0.703756
Epoch 113
Validation auc = 0.707512
Epoch 114
Loss = 1.5920e-02, PNorm = 108.1953, GNorm = 1.6269, lr_0 = 8.3288e-04
Validation auc = 0.712207
Epoch 115
Validation auc = 0.722535
Epoch 116
Validation auc = 0.714554
Epoch 117
Validation auc = 0.723944
Epoch 118
Validation auc = 0.727230
Epoch 119
Loss = 1.1160e-02, PNorm = 108.4163, GNorm = 1.2686, lr_0 = 8.2576e-04
Validation auc = 0.732864
Epoch 120
Validation auc = 0.737559
Epoch 121
Validation auc = 0.741315
Epoch 122
Validation auc = 0.741784
Epoch 123
Validation auc = 0.742254
Epoch 124
Loss = 2.4725e-02, PNorm = 108.6256, GNorm = 0.4148, lr_0 = 8.1870e-04
Validation auc = 0.743662
Epoch 125
Validation auc = 0.733803
Epoch 126
Validation auc = 0.726761
Epoch 127
Validation auc = 0.724883
Epoch 128
Validation auc = 0.724413
Epoch 129
Loss = 4.6193e-03, PNorm = 108.8290, GNorm = 0.6058, lr_0 = 8.1170e-04
Validation auc = 0.720188
Epoch 130
Validation auc = 0.720188
Epoch 131
Validation auc = 0.720188
Epoch 132
Validation auc = 0.720188
Epoch 133
Validation auc = 0.720188
Epoch 134
Loss = 4.5141e-03, PNorm = 108.9807, GNorm = 0.6727, lr_0 = 8.0476e-04
Validation auc = 0.717371
Epoch 135
Validation auc = 0.717840
Epoch 136
Validation auc = 0.710798
Epoch 137
Validation auc = 0.711737
Epoch 138
Validation auc = 0.715493
Epoch 139
Loss = 3.2333e-03, PNorm = 109.0972, GNorm = 0.0972, lr_0 = 7.9788e-04
Validation auc = 0.712676
Epoch 140
Validation auc = 0.715962
Epoch 141
Validation auc = 0.714085
Epoch 142
Validation auc = 0.713146
Epoch 143
Validation auc = 0.722535
Epoch 144
Loss = 3.3675e-03, PNorm = 109.2066, GNorm = 0.1557, lr_0 = 7.9106e-04
Validation auc = 0.717840
Epoch 145
Validation auc = 0.711268
Epoch 146
Validation auc = 0.698122
Epoch 147
Validation auc = 0.691080
Epoch 148
Validation auc = 0.679343
Epoch 149
Loss = 3.9305e-03, PNorm = 109.3096, GNorm = 0.3297, lr_0 = 7.8430e-04
Validation auc = 0.678873
Epoch 150
Validation auc = 0.680751
Epoch 151
Validation auc = 0.677934
Epoch 152
Validation auc = 0.675117
Epoch 153
Validation auc = 0.682160
Epoch 154
Loss = 1.0253e-02, PNorm = 109.4014, GNorm = 1.7488, lr_0 = 7.7759e-04
Validation auc = 0.670892
Epoch 155
Validation auc = 0.680282
Epoch 156
Validation auc = 0.691080
Epoch 157
Validation auc = 0.687793
Epoch 158
Validation auc = 0.686385
Epoch 159
Loss = 3.0371e-02, PNorm = 109.5825, GNorm = 3.8143, lr_0 = 7.7095e-04
Validation auc = 0.679812
Epoch 160
Validation auc = 0.674648
Epoch 161
Validation auc = 0.683099
Epoch 162
Validation auc = 0.681690
Epoch 163
Validation auc = 0.687324
Epoch 164
Loss = 1.3362e-02, PNorm = 109.7610, GNorm = 0.4211, lr_0 = 7.6436e-04
Validation auc = 0.686385
Epoch 165
Validation auc = 0.694366
Epoch 166
Validation auc = 0.707981
Epoch 167
Validation auc = 0.710798
Epoch 168
Validation auc = 0.710329
Epoch 169
Loss = 3.2863e-02, PNorm = 109.9010, GNorm = 1.9377, lr_0 = 7.5782e-04
Validation auc = 0.706573
Epoch 170
Validation auc = 0.705164
Epoch 171
Validation auc = 0.695305
Epoch 172
Validation auc = 0.687324
Epoch 173
Validation auc = 0.684507
Epoch 174
Loss = 2.4902e-03, PNorm = 110.0817, GNorm = 0.1336, lr_0 = 7.5134e-04
Validation auc = 0.678873
Epoch 175
Validation auc = 0.676526
Epoch 176
Validation auc = 0.674178
Epoch 177
Validation auc = 0.676291
Epoch 178
Validation auc = 0.665258
Epoch 179
Loss = 1.3303e-03, PNorm = 110.2017, GNorm = 0.2473, lr_0 = 7.4492e-04
Validation auc = 0.653521
Epoch 180
Validation auc = 0.649296
Epoch 181
Validation auc = 0.647418
Epoch 182
Validation auc = 0.650235
Epoch 183
Validation auc = 0.655869
Epoch 184
Loss = 4.9163e-02, PNorm = 110.2775, GNorm = 2.7623, lr_0 = 7.3855e-04
Validation auc = 0.663380
Epoch 185
Validation auc = 0.678404
Epoch 186
Validation auc = 0.693897
Epoch 187
Validation auc = 0.703286
Epoch 188
Validation auc = 0.705164
Epoch 189
Loss = 3.9059e-03, PNorm = 110.3625, GNorm = 0.4464, lr_0 = 7.3224e-04
Validation auc = 0.705634
Epoch 190
Validation auc = 0.711268
Epoch 191
Validation auc = 0.710798
Epoch 192
Validation auc = 0.711268
Epoch 193
Validation auc = 0.711737
Epoch 194
Loss = 8.3063e-03, PNorm = 110.4268, GNorm = 1.1783, lr_0 = 7.2598e-04
Validation auc = 0.711268
Epoch 195
Validation auc = 0.709859
Epoch 196
Validation auc = 0.709859
Epoch 197
Validation auc = 0.712207
Epoch 198
Validation auc = 0.713146
Epoch 199
Loss = 4.4460e-03, PNorm = 110.4719, GNorm = 0.7564, lr_0 = 7.1977e-04
Validation auc = 0.710329
Epoch 200
Validation auc = 0.712676
Epoch 201
Validation auc = 0.714554
Epoch 202
Validation auc = 0.714554
Epoch 203
Validation auc = 0.715023
Epoch 204
Loss = 1.2229e-03, PNorm = 110.5069, GNorm = 0.0459, lr_0 = 7.1362e-04
Validation auc = 0.716901
Epoch 205
Validation auc = 0.717371
Epoch 206
Validation auc = 0.719718
Epoch 207
Validation auc = 0.721596
Epoch 208
Validation auc = 0.721596
Epoch 209
Loss = 3.0441e-04, PNorm = 110.5256, GNorm = 0.0216, lr_0 = 7.0752e-04
Validation auc = 0.719249
Epoch 210
Validation auc = 0.719249
Epoch 211
Validation auc = 0.718779
Epoch 212
Validation auc = 0.715962
Epoch 213
Validation auc = 0.716432
Epoch 214
Loss = 1.7368e-04, PNorm = 110.5397, GNorm = 0.0150, lr_0 = 7.0147e-04
Validation auc = 0.715962
Epoch 215
Validation auc = 0.714085
Epoch 216
Validation auc = 0.713146
Epoch 217
Validation auc = 0.715023
Epoch 218
Validation auc = 0.715493
Epoch 219
Loss = 2.1965e-04, PNorm = 110.5553, GNorm = 0.0277, lr_0 = 6.9548e-04
Validation auc = 0.715962
Epoch 220
Validation auc = 0.715962
Epoch 221
Validation auc = 0.716901
Epoch 222
Validation auc = 0.723005
Epoch 223
Validation auc = 0.727230
Epoch 224
Loss = 5.8684e-03, PNorm = 110.5771, GNorm = 0.0044, lr_0 = 6.8953e-04
Validation auc = 0.724413
Epoch 225
Validation auc = 0.724883
Epoch 226
Validation auc = 0.719718
Epoch 227
Validation auc = 0.717371
Epoch 228
Validation auc = 0.722066
Epoch 229
Loss = 2.9615e-02, PNorm = 110.6150, GNorm = 0.1058, lr_0 = 6.8364e-04
Validation auc = 0.705164
Epoch 230
Validation auc = 0.707981
Epoch 231
Validation auc = 0.713146
Epoch 232
Validation auc = 0.717840
Epoch 233
Validation auc = 0.722066
Epoch 234
Loss = 9.0453e-03, PNorm = 110.6979, GNorm = 0.0071, lr_0 = 6.7779e-04
Validation auc = 0.722066
Epoch 235
Validation auc = 0.723944
Epoch 236
Validation auc = 0.719718
Epoch 237
Validation auc = 0.715023
Epoch 238
Validation auc = 0.712207
Epoch 239
Loss = 7.1493e-04, PNorm = 110.8063, GNorm = 0.0215, lr_0 = 6.7200e-04
Validation auc = 0.715493
Epoch 240
Validation auc = 0.704225
Epoch 241
Validation auc = 0.705164
Epoch 242
Validation auc = 0.707981
Epoch 243
Validation auc = 0.709390
Epoch 244
Loss = 2.4090e-03, PNorm = 110.8941, GNorm = 0.0476, lr_0 = 6.6625e-04
Validation auc = 0.718310
Epoch 245
Validation auc = 0.718310
Epoch 246
Validation auc = 0.729108
Epoch 247
Validation auc = 0.737089
Epoch 248
Validation auc = 0.746948
Epoch 249
Loss = 2.6251e-03, PNorm = 111.0068, GNorm = 0.2075, lr_0 = 6.6056e-04
Validation auc = 0.749296
Epoch 250
Validation auc = 0.746009
Epoch 251
Validation auc = 0.746479
Epoch 252
Validation auc = 0.743192
Epoch 253
Validation auc = 0.742254
Epoch 254
Loss = 3.6083e-04, PNorm = 111.1020, GNorm = 0.0169, lr_0 = 6.5491e-04
Validation auc = 0.741315
Epoch 255
Validation auc = 0.742723
Epoch 256
Validation auc = 0.743192
Epoch 257
Validation auc = 0.741315
Epoch 258
Validation auc = 0.742723
Epoch 259
Loss = 2.7431e-04, PNorm = 111.1757, GNorm = 0.0173, lr_0 = 6.4931e-04
Validation auc = 0.740845
Epoch 260
Validation auc = 0.737559
Epoch 261
Validation auc = 0.735211
Epoch 262
Validation auc = 0.733803
Epoch 263
Validation auc = 0.732864
Epoch 264
Loss = 1.7851e-04, PNorm = 111.2085, GNorm = 0.0076, lr_0 = 6.4376e-04
Validation auc = 0.732864
Epoch 265
Validation auc = 0.731925
Epoch 266
Validation auc = 0.730516
Epoch 267
Validation auc = 0.729577
Epoch 268
Validation auc = 0.729108
Epoch 269
Loss = 4.3450e-04, PNorm = 111.2219, GNorm = 0.0028, lr_0 = 6.3826e-04
Validation auc = 0.728169
Epoch 270
Validation auc = 0.729108
Epoch 271
Validation auc = 0.729577
Epoch 272
Validation auc = 0.731455
Epoch 273
Validation auc = 0.730516
Epoch 274
Loss = 3.0323e-04, PNorm = 111.2309, GNorm = 0.0341, lr_0 = 6.3280e-04
Validation auc = 0.730516
Epoch 275
Validation auc = 0.730986
Epoch 276
Validation auc = 0.730986
Epoch 277
Validation auc = 0.730986
Epoch 278
Validation auc = 0.730986
Epoch 279
Loss = 2.7077e-04, PNorm = 111.2370, GNorm = 0.0510, lr_0 = 6.2739e-04
Validation auc = 0.730986
Epoch 280
Validation auc = 0.730516
Epoch 281
Validation auc = 0.731455
Epoch 282
Validation auc = 0.731455
Epoch 283
Validation auc = 0.731455
Epoch 284
Loss = 1.1619e-04, PNorm = 111.2417, GNorm = 0.0070, lr_0 = 6.2203e-04
Validation auc = 0.732864
Epoch 285
Validation auc = 0.732394
Epoch 286
Validation auc = 0.732394
Epoch 287
Validation auc = 0.731925
Epoch 288
Validation auc = 0.730986
Epoch 289
Loss = 8.2173e-04, PNorm = 111.2448, GNorm = 0.0105, lr_0 = 6.1671e-04
Validation auc = 0.731455
Epoch 290
Validation auc = 0.731925
Epoch 291
Validation auc = 0.732394
Epoch 292
Validation auc = 0.734272
Epoch 293
Validation auc = 0.734742
Epoch 294
Loss = 2.2214e-04, PNorm = 111.2479, GNorm = 0.0128, lr_0 = 6.1144e-04
Validation auc = 0.734272
Epoch 295
Validation auc = 0.735211
Epoch 296
Validation auc = 0.736150
Epoch 297
Validation auc = 0.735681
Epoch 298
Validation auc = 0.735681
Epoch 299
Loss = 1.3434e-04, PNorm = 111.2509, GNorm = 0.0196, lr_0 = 6.0621e-04
Validation auc = 0.735681
Model 0 best validation auc = 0.749296 on epoch 249
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.880795
Ensemble test auc = 0.880795
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.693427
Epoch 1
Validation auc = 0.677934
Epoch 2
Validation auc = 0.693897
Epoch 3
Validation auc = 0.701408
Epoch 4
Loss = 5.8204e-01, PNorm = 100.5522, GNorm = 3.4892, lr_0 = 6.5000e-04
Validation auc = 0.616901
Epoch 5
Validation auc = 0.667606
Epoch 6
Validation auc = 0.631455
Epoch 7
Validation auc = 0.657746
Epoch 8
Validation auc = 0.628169
Epoch 9
Loss = 5.1533e-01, PNorm = 100.9220, GNorm = 2.8204, lr_0 = 9.9743e-04
Validation auc = 0.621127
Epoch 10
Validation auc = 0.621127
Epoch 11
Validation auc = 0.669014
Epoch 12
Validation auc = 0.668545
Epoch 13
Validation auc = 0.658685
Epoch 14
Loss = 3.0047e-01, PNorm = 101.3125, GNorm = 1.8193, lr_0 = 9.8890e-04
Validation auc = 0.653991
Epoch 15
Validation auc = 0.629108
Epoch 16
Validation auc = 0.614085
Epoch 17
Validation auc = 0.615493
Epoch 18
Validation auc = 0.670892
Epoch 19
Loss = 4.0509e-01, PNorm = 101.6357, GNorm = 8.1922, lr_0 = 9.8045e-04
Validation auc = 0.669014
Epoch 20
Validation auc = 0.663380
Epoch 21
Validation auc = 0.690610
Epoch 22
Validation auc = 0.670892
Epoch 23
Validation auc = 0.685446
Epoch 24
Loss = 2.8365e-01, PNorm = 101.9393, GNorm = 5.8866, lr_0 = 9.7207e-04
Validation auc = 0.687793
Epoch 25
Validation auc = 0.669484
Epoch 26
Validation auc = 0.671831
Epoch 27
Validation auc = 0.675117
Epoch 28
Validation auc = 0.659155
Epoch 29
Loss = 3.5404e-01, PNorm = 102.2169, GNorm = 3.2944, lr_0 = 9.6376e-04
Validation auc = 0.667136
Epoch 30
Validation auc = 0.689671
Epoch 31
Validation auc = 0.684507
Epoch 32
Validation auc = 0.683568
Epoch 33
Validation auc = 0.691080
Epoch 34
Loss = 1.9213e-01, PNorm = 102.5109, GNorm = 1.9816, lr_0 = 9.5552e-04
Validation auc = 0.688732
Epoch 35
Validation auc = 0.680751
Epoch 36
Validation auc = 0.656808
Epoch 37
Validation auc = 0.657746
Epoch 38
Validation auc = 0.673709
Epoch 39
Loss = 2.7598e-01, PNorm = 102.7885, GNorm = 6.0736, lr_0 = 9.4735e-04
Validation auc = 0.666197
Epoch 40
Validation auc = 0.658685
Epoch 41
Validation auc = 0.647887
Epoch 42
Validation auc = 0.678404
Epoch 43
Validation auc = 0.697653
Epoch 44
Loss = 3.0237e-01, PNorm = 103.0777, GNorm = 2.6784, lr_0 = 9.3925e-04
Validation auc = 0.698592
Epoch 45
Validation auc = 0.697653
Epoch 46
Validation auc = 0.700939
Epoch 47
Validation auc = 0.700000
Epoch 48
Validation auc = 0.696244
Epoch 49
Loss = 2.1463e-01, PNorm = 103.3931, GNorm = 2.8579, lr_0 = 9.3122e-04
Validation auc = 0.691080
Epoch 50
Validation auc = 0.691080
Epoch 51
Validation auc = 0.684507
Epoch 52
Validation auc = 0.680282
Epoch 53
Validation auc = 0.674178
Epoch 54
Loss = 1.0104e-01, PNorm = 103.7157, GNorm = 1.9065, lr_0 = 9.2326e-04
Validation auc = 0.677465
Epoch 55
Validation auc = 0.686385
Epoch 56
Validation auc = 0.692019
Epoch 57
Validation auc = 0.701878
Epoch 58
Validation auc = 0.700469
Epoch 59
Loss = 1.5757e-01, PNorm = 104.0501, GNorm = 4.8091, lr_0 = 9.1537e-04
Validation auc = 0.693897
Epoch 60
Validation auc = 0.684038
Epoch 61
Validation auc = 0.689671
Epoch 62
Validation auc = 0.712676
Epoch 63
Validation auc = 0.724413
Epoch 64
Loss = 8.8647e-02, PNorm = 104.3922, GNorm = 1.5331, lr_0 = 9.0754e-04
Validation auc = 0.721596
Epoch 65
Validation auc = 0.716432
Epoch 66
Validation auc = 0.718779
Epoch 67
Validation auc = 0.725352
Epoch 68
Validation auc = 0.728169
Epoch 69
Loss = 4.0838e-02, PNorm = 104.7062, GNorm = 1.5923, lr_0 = 8.9978e-04
Validation auc = 0.730986
Epoch 70
Validation auc = 0.727230
Epoch 71
Validation auc = 0.722535
Epoch 72
Validation auc = 0.711268
Epoch 73
Validation auc = 0.694366
Epoch 74
Loss = 6.8517e-02, PNorm = 105.0066, GNorm = 3.1309, lr_0 = 8.9209e-04
Validation auc = 0.676995
Epoch 75
Validation auc = 0.679343
Epoch 76
Validation auc = 0.682629
Epoch 77
Validation auc = 0.680282
Epoch 78
Validation auc = 0.688263
Epoch 79
Loss = 1.2011e-01, PNorm = 105.3230, GNorm = 1.0297, lr_0 = 8.8447e-04
Validation auc = 0.692958
Epoch 80
Validation auc = 0.698122
Epoch 81
Validation auc = 0.710329
Epoch 82
Validation auc = 0.715493
Epoch 83
Validation auc = 0.713146
Epoch 84
Loss = 1.7252e-01, PNorm = 105.6653, GNorm = 6.1105, lr_0 = 8.7691e-04
Validation auc = 0.698122
Epoch 85
Validation auc = 0.687793
Epoch 86
Validation auc = 0.697653
Epoch 87
Validation auc = 0.686385
Epoch 88
Validation auc = 0.695305
Epoch 89
Loss = 1.6500e-01, PNorm = 106.0666, GNorm = 4.4352, lr_0 = 8.6941e-04
Validation auc = 0.706103
Epoch 90
Validation auc = 0.708920
Epoch 91
Validation auc = 0.709390
Epoch 92
Validation auc = 0.700939
Epoch 93
Validation auc = 0.704695
Epoch 94
Loss = 3.9700e-02, PNorm = 106.4356, GNorm = 1.1136, lr_0 = 8.6198e-04
Validation auc = 0.702347
Epoch 95
Validation auc = 0.698122
Epoch 96
Validation auc = 0.689671
Epoch 97
Validation auc = 0.684977
Epoch 98
Validation auc = 0.684977
Epoch 99
Loss = 3.4370e-02, PNorm = 106.7373, GNorm = 1.5519, lr_0 = 8.5461e-04
Validation auc = 0.676995
Epoch 100
Validation auc = 0.732864
Epoch 101
Validation auc = 0.759624
Epoch 102
Validation auc = 0.765258
Epoch 103
Validation auc = 0.763850
Epoch 104
Loss = 4.7899e-02, PNorm = 107.0496, GNorm = 1.2909, lr_0 = 8.4730e-04
Validation auc = 0.740845
Epoch 105
Validation auc = 0.746479
Epoch 106
Validation auc = 0.740845
Epoch 107
Validation auc = 0.724413
Epoch 108
Validation auc = 0.706573
Epoch 109
Loss = 7.8691e-02, PNorm = 107.3318, GNorm = 4.3654, lr_0 = 8.4006e-04
Validation auc = 0.707042
Epoch 110
Validation auc = 0.728169
Epoch 111
Validation auc = 0.737559
Epoch 112
Validation auc = 0.744601
Epoch 113
Validation auc = 0.747887
Epoch 114
Loss = 7.0954e-02, PNorm = 107.6243, GNorm = 2.8442, lr_0 = 8.3288e-04
Validation auc = 0.761033
Epoch 115
Validation auc = 0.753991
Epoch 116
Validation auc = 0.747418
Epoch 117
Validation auc = 0.741784
Epoch 118
Validation auc = 0.711268
Epoch 119
Loss = 1.7298e-02, PNorm = 107.9632, GNorm = 1.1699, lr_0 = 8.2576e-04
Validation auc = 0.678404
Epoch 120
Validation auc = 0.729577
Epoch 121
Validation auc = 0.743192
Epoch 122
Validation auc = 0.734742
Epoch 123
Validation auc = 0.733333
Epoch 124
Loss = 6.9475e-02, PNorm = 108.2300, GNorm = 1.7145, lr_0 = 8.1870e-04
Validation auc = 0.736620
Epoch 125
Validation auc = 0.738967
Epoch 126
Validation auc = 0.738498
Epoch 127
Validation auc = 0.738967
Epoch 128
Validation auc = 0.728169
Epoch 129
Loss = 2.6393e-02, PNorm = 108.4413, GNorm = 0.2499, lr_0 = 8.1170e-04
Validation auc = 0.730516
Epoch 130
Validation auc = 0.722066
Epoch 131
Validation auc = 0.709859
Epoch 132
Validation auc = 0.708920
Epoch 133
Validation auc = 0.715962
Epoch 134
Loss = 4.7727e-02, PNorm = 108.6805, GNorm = 1.2781, lr_0 = 8.0476e-04
Validation auc = 0.719718
Epoch 135
Validation auc = 0.724883
Epoch 136
Validation auc = 0.722535
Epoch 137
Validation auc = 0.724413
Epoch 138
Validation auc = 0.726761
Epoch 139
Loss = 9.8534e-02, PNorm = 108.9710, GNorm = 4.4996, lr_0 = 7.9788e-04
Validation auc = 0.733803
Epoch 140
Validation auc = 0.743662
Epoch 141
Validation auc = 0.744601
Epoch 142
Validation auc = 0.741784
Epoch 143
Validation auc = 0.738498
Epoch 144
Loss = 3.8286e-02, PNorm = 109.2550, GNorm = 0.7789, lr_0 = 7.9106e-04
Validation auc = 0.737089
Epoch 145
Validation auc = 0.736150
Epoch 146
Validation auc = 0.730516
Epoch 147
Validation auc = 0.729108
Epoch 148
Validation auc = 0.726761
Epoch 149
Loss = 1.2487e-02, PNorm = 109.4818, GNorm = 1.2980, lr_0 = 7.8430e-04
Validation auc = 0.731925
Epoch 150
Validation auc = 0.733803
Epoch 151
Validation auc = 0.732394
Epoch 152
Validation auc = 0.725352
Epoch 153
Validation auc = 0.719249
Epoch 154
Loss = 6.0503e-02, PNorm = 109.6535, GNorm = 1.1413, lr_0 = 7.7759e-04
Validation auc = 0.749765
Epoch 155
Validation auc = 0.739437
Epoch 156
Validation auc = 0.739906
Epoch 157
Validation auc = 0.750235
Epoch 158
Validation auc = 0.762911
Epoch 159
Loss = 5.1399e-02, PNorm = 109.8753, GNorm = 1.2728, lr_0 = 7.7095e-04
Validation auc = 0.743662
Epoch 160
Validation auc = 0.739437
Epoch 161
Validation auc = 0.740376
Epoch 162
Validation auc = 0.742254
Epoch 163
Validation auc = 0.741784
Epoch 164
Loss = 4.1796e-02, PNorm = 110.0965, GNorm = 1.4399, lr_0 = 7.6436e-04
Validation auc = 0.731455
Epoch 165
Validation auc = 0.730047
Epoch 166
Validation auc = 0.724413
Epoch 167
Validation auc = 0.721127
Epoch 168
Validation auc = 0.715962
Epoch 169
Loss = 2.7672e-03, PNorm = 110.3094, GNorm = 0.0522, lr_0 = 7.5782e-04
Validation auc = 0.715023
Epoch 170
Validation auc = 0.718310
Epoch 171
Validation auc = 0.730986
Epoch 172
Validation auc = 0.741315
Epoch 173
Validation auc = 0.743662
Epoch 174
Loss = 4.8168e-03, PNorm = 110.4391, GNorm = 0.2857, lr_0 = 7.5134e-04
Validation auc = 0.742723
Epoch 175
Validation auc = 0.744131
Epoch 176
Validation auc = 0.741784
Epoch 177
Validation auc = 0.739906
Epoch 178
Validation auc = 0.735211
Epoch 179
Loss = 4.6042e-03, PNorm = 110.5584, GNorm = 0.0605, lr_0 = 7.4492e-04
Validation auc = 0.738028
Epoch 180
Validation auc = 0.732394
Epoch 181
Validation auc = 0.733333
Epoch 182
Validation auc = 0.734272
Epoch 183
Validation auc = 0.733333
Epoch 184
Loss = 8.3842e-03, PNorm = 110.6637, GNorm = 0.1484, lr_0 = 7.3855e-04
Validation auc = 0.733333
Epoch 185
Validation auc = 0.738967
Epoch 186
Validation auc = 0.745070
Epoch 187
Validation auc = 0.738498
Epoch 188
Validation auc = 0.730516
Epoch 189
Loss = 1.3594e-02, PNorm = 110.7409, GNorm = 0.5818, lr_0 = 7.3224e-04
Validation auc = 0.726291
Epoch 190
Validation auc = 0.727230
Epoch 191
Validation auc = 0.721127
Epoch 192
Validation auc = 0.724883
Epoch 193
Validation auc = 0.722066
Epoch 194
Loss = 1.9722e-02, PNorm = 110.8272, GNorm = 2.7021, lr_0 = 7.2598e-04
Validation auc = 0.718310
Epoch 195
Validation auc = 0.725352
Epoch 196
Validation auc = 0.730047
Epoch 197
Validation auc = 0.723944
Epoch 198
Validation auc = 0.722066
Epoch 199
Loss = 1.1779e-02, PNorm = 110.9271, GNorm = 0.3997, lr_0 = 7.1977e-04
Validation auc = 0.718310
Epoch 200
Validation auc = 0.719249
Epoch 201
Validation auc = 0.717840
Epoch 202
Validation auc = 0.718310
Epoch 203
Validation auc = 0.719718
Epoch 204
Loss = 2.3291e-03, PNorm = 111.0091, GNorm = 0.2447, lr_0 = 7.1362e-04
Validation auc = 0.726291
Epoch 205
Validation auc = 0.728169
Epoch 206
Validation auc = 0.729108
Epoch 207
Validation auc = 0.732864
Epoch 208
Validation auc = 0.734742
Epoch 209
Loss = 5.4146e-04, PNorm = 111.0705, GNorm = 0.0341, lr_0 = 7.0752e-04
Validation auc = 0.735681
Epoch 210
Validation auc = 0.737559
Epoch 211
Validation auc = 0.738967
Epoch 212
Validation auc = 0.738967
Epoch 213
Validation auc = 0.738498
Epoch 214
Loss = 1.1177e-03, PNorm = 111.1088, GNorm = 0.1411, lr_0 = 7.0147e-04
Validation auc = 0.738028
Epoch 215
Validation auc = 0.738967
Epoch 216
Validation auc = 0.741315
Epoch 217
Validation auc = 0.742254
Epoch 218
Validation auc = 0.739906
Epoch 219
Loss = 2.2347e-03, PNorm = 111.1369, GNorm = 0.1648, lr_0 = 6.9548e-04
Validation auc = 0.739437
Epoch 220
Validation auc = 0.740845
Epoch 221
Validation auc = 0.740376
Epoch 222
Validation auc = 0.738498
Epoch 223
Validation auc = 0.737559
Epoch 224
Loss = 1.8480e-03, PNorm = 111.1702, GNorm = 0.0512, lr_0 = 6.8953e-04
Validation auc = 0.738967
Epoch 225
Validation auc = 0.738967
Epoch 226
Validation auc = 0.738967
Epoch 227
Validation auc = 0.731455
Epoch 228
Validation auc = 0.731925
Epoch 229
Loss = 1.5921e-04, PNorm = 111.2003, GNorm = 0.0146, lr_0 = 6.8364e-04
Validation auc = 0.730047
Epoch 230
Validation auc = 0.729577
Epoch 231
Validation auc = 0.729108
Epoch 232
Validation auc = 0.730047
Epoch 233
Validation auc = 0.730986
Epoch 234
Loss = 7.1984e-05, PNorm = 111.2308, GNorm = 0.0036, lr_0 = 6.7779e-04
Validation auc = 0.728638
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.729108
Epoch 237
Validation auc = 0.727700
Epoch 238
Validation auc = 0.730047
Epoch 239
Loss = 2.1238e-04, PNorm = 111.2482, GNorm = 0.0108, lr_0 = 6.7200e-04
Validation auc = 0.730986
Epoch 240
Validation auc = 0.729108
Epoch 241
Validation auc = 0.730047
Epoch 242
Validation auc = 0.728638
Epoch 243
Validation auc = 0.727700
Epoch 244
Loss = 2.2822e-04, PNorm = 111.2763, GNorm = 0.0073, lr_0 = 6.6625e-04
Validation auc = 0.728638
Epoch 245
Validation auc = 0.725822
Epoch 246
Validation auc = 0.726291
Epoch 247
Validation auc = 0.727230
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 2.0329e-04, PNorm = 111.2930, GNorm = 0.0028, lr_0 = 6.6056e-04
Validation auc = 0.727230
Epoch 250
Validation auc = 0.728169
Epoch 251
Validation auc = 0.729577
Epoch 252
Validation auc = 0.730986
Epoch 253
Validation auc = 0.735211
Epoch 254
Loss = 1.7670e-03, PNorm = 111.3083, GNorm = 0.0643, lr_0 = 6.5491e-04
Validation auc = 0.737089
Epoch 255
Validation auc = 0.738028
Epoch 256
Validation auc = 0.739906
Epoch 257
Validation auc = 0.743662
Epoch 258
Validation auc = 0.741315
Epoch 259
Loss = 1.1492e-04, PNorm = 111.3459, GNorm = 0.0011, lr_0 = 6.4931e-04
Validation auc = 0.742254
Epoch 260
Validation auc = 0.743662
Epoch 261
Validation auc = 0.743192
Epoch 262
Validation auc = 0.744601
Epoch 263
Validation auc = 0.745540
Epoch 264
Loss = 2.9037e-05, PNorm = 111.3633, GNorm = 0.0052, lr_0 = 6.4376e-04
Validation auc = 0.745540
Epoch 265
Validation auc = 0.745070
Epoch 266
Validation auc = 0.744131
Epoch 267
Validation auc = 0.744601
Epoch 268
Validation auc = 0.744131
Epoch 269
Loss = 3.4315e-05, PNorm = 111.3696, GNorm = 0.0025, lr_0 = 6.3826e-04
Validation auc = 0.743662
Epoch 270
Validation auc = 0.742254
Epoch 271
Validation auc = 0.744131
Epoch 272
Validation auc = 0.744601
Epoch 273
Validation auc = 0.743662
Epoch 274
Loss = 4.1473e-04, PNorm = 111.3731, GNorm = 0.0952, lr_0 = 6.3280e-04
Validation auc = 0.744601
Epoch 275
Validation auc = 0.743192
Epoch 276
Validation auc = 0.742723
Epoch 277
Validation auc = 0.742254
Epoch 278
Validation auc = 0.742254
Epoch 279
Loss = 2.3258e-05, PNorm = 111.3763, GNorm = 0.0014, lr_0 = 6.2739e-04
Validation auc = 0.742254
Epoch 280
Validation auc = 0.742254
Epoch 281
Validation auc = 0.742254
Epoch 282
Validation auc = 0.741784
Epoch 283
Validation auc = 0.741784
Epoch 284
Loss = 1.2492e-04, PNorm = 111.3788, GNorm = 0.0019, lr_0 = 6.2203e-04
Validation auc = 0.741784
Epoch 285
Validation auc = 0.741315
Epoch 286
Validation auc = 0.740845
Epoch 287
Validation auc = 0.741315
Epoch 288
Validation auc = 0.738498
Epoch 289
Loss = 9.1508e-04, PNorm = 111.3782, GNorm = 0.2537, lr_0 = 6.1671e-04
Validation auc = 0.741784
Epoch 290
Validation auc = 0.734742
Epoch 291
Validation auc = 0.729108
Epoch 292
Validation auc = 0.722535
Epoch 293
Validation auc = 0.718779
Epoch 294
Loss = 1.1405e-03, PNorm = 111.3978, GNorm = 0.0055, lr_0 = 6.1144e-04
Validation auc = 0.721596
Epoch 295
Validation auc = 0.723474
Epoch 296
Validation auc = 0.723944
Epoch 297
Validation auc = 0.723474
Epoch 298
Validation auc = 0.722066
Epoch 299
Loss = 6.4375e-05, PNorm = 111.4193, GNorm = 0.0068, lr_0 = 6.0621e-04
Validation auc = 0.721596
Model 0 best validation auc = 0.765258 on epoch 102
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.860325
Ensemble test auc = 0.860325
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.642723
Epoch 1
Validation auc = 0.540376
Epoch 2
Validation auc = 0.604695
Epoch 3
Validation auc = 0.653521
Epoch 4
Loss = 6.1136e-01, PNorm = 100.5442, GNorm = 6.2699, lr_0 = 6.5000e-04
Validation auc = 0.612207
Epoch 5
Validation auc = 0.649765
Epoch 6
Validation auc = 0.643192
Epoch 7
Validation auc = 0.694836
Epoch 8
Validation auc = 0.701878
Epoch 9
Loss = 4.8665e-01, PNorm = 100.8965, GNorm = 1.6772, lr_0 = 9.9743e-04
Validation auc = 0.693427
Epoch 10
Validation auc = 0.688263
Epoch 11
Validation auc = 0.693897
Epoch 12
Validation auc = 0.706103
Epoch 13
Validation auc = 0.723474
Epoch 14
Loss = 4.0409e-01, PNorm = 101.2607, GNorm = 1.9988, lr_0 = 9.8890e-04
Validation auc = 0.745070
Epoch 15
Validation auc = 0.732864
Epoch 16
Validation auc = 0.705634
Epoch 17
Validation auc = 0.698122
Epoch 18
Validation auc = 0.720188
Epoch 19
Loss = 2.3995e-01, PNorm = 101.5917, GNorm = 1.8567, lr_0 = 9.8045e-04
Validation auc = 0.726761
Epoch 20
Validation auc = 0.738028
Epoch 21
Validation auc = 0.731455
Epoch 22
Validation auc = 0.687324
Epoch 23
Validation auc = 0.656338
Epoch 24
Loss = 2.9835e-01, PNorm = 101.8948, GNorm = 6.2006, lr_0 = 9.7207e-04
Validation auc = 0.687324
Epoch 25
Validation auc = 0.696714
Epoch 26
Validation auc = 0.703756
Epoch 27
Validation auc = 0.708451
Epoch 28
Validation auc = 0.706103
Epoch 29
Loss = 3.2333e-01, PNorm = 102.1847, GNorm = 5.8056, lr_0 = 9.6376e-04
Validation auc = 0.712207
Epoch 30
Validation auc = 0.707981
Epoch 31
Validation auc = 0.706103
Epoch 32
Validation auc = 0.702817
Epoch 33
Validation auc = 0.699531
Epoch 34
Loss = 1.5647e-01, PNorm = 102.4736, GNorm = 8.3207, lr_0 = 9.5552e-04
Validation auc = 0.677934
Epoch 35
Validation auc = 0.680282
Epoch 36
Validation auc = 0.702347
Epoch 37
Validation auc = 0.713615
Epoch 38
Validation auc = 0.711737
Epoch 39
Loss = 1.2896e-01, PNorm = 102.7893, GNorm = 6.6755, lr_0 = 9.4735e-04
Validation auc = 0.714085
Epoch 40
Validation auc = 0.718779
Epoch 41
Validation auc = 0.713146
Epoch 42
Validation auc = 0.704225
Epoch 43
Validation auc = 0.691080
Epoch 44
Loss = 1.9453e-01, PNorm = 103.1342, GNorm = 8.9502, lr_0 = 9.3925e-04
Validation auc = 0.677934
Epoch 45
Validation auc = 0.676526
Epoch 46
Validation auc = 0.673709
Epoch 47
Validation auc = 0.679343
Epoch 48
Validation auc = 0.681690
Epoch 49
Loss = 1.7512e-01, PNorm = 103.4841, GNorm = 5.5259, lr_0 = 9.3122e-04
Validation auc = 0.679812
Epoch 50
Validation auc = 0.727230
Epoch 51
Validation auc = 0.745070
Epoch 52
Validation auc = 0.749765
Epoch 53
Validation auc = 0.728169
Epoch 54
Loss = 1.3193e-01, PNorm = 103.8764, GNorm = 1.3051, lr_0 = 9.2326e-04
Validation auc = 0.707512
Epoch 55
Validation auc = 0.690610
Epoch 56
Validation auc = 0.684977
Epoch 57
Validation auc = 0.689202
Epoch 58
Validation auc = 0.711268
Epoch 59
Loss = 2.2481e-01, PNorm = 104.3365, GNorm = 9.3693, lr_0 = 9.1537e-04
Validation auc = 0.716432
Epoch 60
Validation auc = 0.720657
Epoch 61
Validation auc = 0.712207
Epoch 62
Validation auc = 0.692019
Epoch 63
Validation auc = 0.685446
Epoch 64
Loss = 4.1122e-02, PNorm = 104.7414, GNorm = 1.0357, lr_0 = 9.0754e-04
Validation auc = 0.683099
Epoch 65
Validation auc = 0.699531
Epoch 66
Validation auc = 0.700000
Epoch 67
Validation auc = 0.701878
Epoch 68
Validation auc = 0.700469
Epoch 69
Loss = 1.3212e-01, PNorm = 105.1160, GNorm = 3.5122, lr_0 = 8.9978e-04
Validation auc = 0.700469
Epoch 70
Validation auc = 0.702817
Epoch 71
Validation auc = 0.699061
Epoch 72
Validation auc = 0.700939
Epoch 73
Validation auc = 0.697183
Epoch 74
Loss = 2.1964e-02, PNorm = 105.4642, GNorm = 1.4167, lr_0 = 8.9209e-04
Validation auc = 0.678404
Epoch 75
Validation auc = 0.686385
Epoch 76
Validation auc = 0.693427
Epoch 77
Validation auc = 0.716901
Epoch 78
Validation auc = 0.706573
Epoch 79
Loss = 7.7927e-02, PNorm = 105.8109, GNorm = 2.5904, lr_0 = 8.8447e-04
Validation auc = 0.691549
Epoch 80
Validation auc = 0.736150
Epoch 81
Validation auc = 0.738498
Epoch 82
Validation auc = 0.726761
Epoch 83
Validation auc = 0.734272
Epoch 84
Loss = 7.1622e-02, PNorm = 106.1435, GNorm = 2.8916, lr_0 = 8.7691e-04
Validation auc = 0.738967
Epoch 85
Validation auc = 0.743662
Epoch 86
Validation auc = 0.754930
Epoch 87
Validation auc = 0.747418
Epoch 88
Validation auc = 0.721596
Epoch 89
Loss = 6.3088e-02, PNorm = 106.4266, GNorm = 3.6592, lr_0 = 8.6941e-04
Validation auc = 0.715023
Epoch 90
Validation auc = 0.723944
Epoch 91
Validation auc = 0.730516
Epoch 92
Validation auc = 0.726761
Epoch 93
Validation auc = 0.726761
Epoch 94
Loss = 1.7276e-02, PNorm = 106.6828, GNorm = 0.5526, lr_0 = 8.6198e-04
Validation auc = 0.720657
Epoch 95
Validation auc = 0.711268
Epoch 96
Validation auc = 0.714085
Epoch 97
Validation auc = 0.719718
Epoch 98
Validation auc = 0.721596
Epoch 99
Loss = 2.5414e-02, PNorm = 106.8904, GNorm = 2.8479, lr_0 = 8.5461e-04
Validation auc = 0.714085
Epoch 100
Validation auc = 0.717840
Epoch 101
Validation auc = 0.703756
Epoch 102
Validation auc = 0.717371
Epoch 103
Validation auc = 0.722535
Epoch 104
Loss = 1.4112e-01, PNorm = 107.1460, GNorm = 7.2007, lr_0 = 8.4730e-04
Validation auc = 0.720657
Epoch 105
Validation auc = 0.714085
Epoch 106
Validation auc = 0.687793
Epoch 107
Validation auc = 0.702817
Epoch 108
Validation auc = 0.698122
Epoch 109
Loss = 7.5061e-02, PNorm = 107.5024, GNorm = 0.4488, lr_0 = 8.4006e-04
Validation auc = 0.690610
Epoch 110
Validation auc = 0.677934
Epoch 111
Validation auc = 0.692488
Epoch 112
Validation auc = 0.676056
Epoch 113
Validation auc = 0.670892
Epoch 114
Loss = 1.5646e-02, PNorm = 107.9158, GNorm = 0.2274, lr_0 = 8.3288e-04
Validation auc = 0.669014
Epoch 115
Validation auc = 0.699061
Epoch 116
Validation auc = 0.696714
Epoch 117
Validation auc = 0.698592
Epoch 118
Validation auc = 0.698592
Epoch 119
Loss = 3.6853e-02, PNorm = 108.3092, GNorm = 2.0364, lr_0 = 8.2576e-04
Validation auc = 0.703286
Epoch 120
Validation auc = 0.710798
Epoch 121
Validation auc = 0.715493
Epoch 122
Validation auc = 0.720657
Epoch 123
Validation auc = 0.723474
Epoch 124
Loss = 9.9970e-02, PNorm = 108.5902, GNorm = 2.1456, lr_0 = 8.1870e-04
Validation auc = 0.725352
Epoch 125
Validation auc = 0.723944
Epoch 126
Validation auc = 0.727230
Epoch 127
Validation auc = 0.726761
Epoch 128
Validation auc = 0.721596
Epoch 129
Loss = 2.4253e-02, PNorm = 108.7943, GNorm = 1.9055, lr_0 = 8.1170e-04
Validation auc = 0.725352
Epoch 130
Validation auc = 0.731925
Epoch 131
Validation auc = 0.735681
Epoch 132
Validation auc = 0.739437
Epoch 133
Validation auc = 0.735211
Epoch 134
Loss = 1.5703e-02, PNorm = 108.9426, GNorm = 1.6318, lr_0 = 8.0476e-04
Validation auc = 0.739906
Epoch 135
Validation auc = 0.738967
Epoch 136
Validation auc = 0.738028
Epoch 137
Validation auc = 0.728169
Epoch 138
Validation auc = 0.727700
Epoch 139
Loss = 1.1168e-02, PNorm = 109.0571, GNorm = 0.1479, lr_0 = 7.9788e-04
Validation auc = 0.726761
Epoch 140
Validation auc = 0.722066
Epoch 141
Validation auc = 0.716901
Epoch 142
Validation auc = 0.712207
Epoch 143
Validation auc = 0.707042
Epoch 144
Loss = 2.7777e-02, PNorm = 109.1854, GNorm = 2.4421, lr_0 = 7.9106e-04
Validation auc = 0.705164
Epoch 145
Validation auc = 0.699061
Epoch 146
Validation auc = 0.699061
Epoch 147
Validation auc = 0.700939
Epoch 148
Validation auc = 0.705164
Epoch 149
Loss = 2.9842e-03, PNorm = 109.3093, GNorm = 0.4060, lr_0 = 7.8430e-04
Validation auc = 0.706573
Epoch 150
Validation auc = 0.709859
Epoch 151
Validation auc = 0.711737
Epoch 152
Validation auc = 0.712676
Epoch 153
Validation auc = 0.714085
Epoch 154
Loss = 2.5887e-03, PNorm = 109.4156, GNorm = 0.5403, lr_0 = 7.7759e-04
Validation auc = 0.707512
Epoch 155
Validation auc = 0.704695
Epoch 156
Validation auc = 0.705164
Epoch 157
Validation auc = 0.707512
Epoch 158
Validation auc = 0.711268
Epoch 159
Loss = 9.2895e-03, PNorm = 109.5380, GNorm = 0.6258, lr_0 = 7.7095e-04
Validation auc = 0.713615
Epoch 160
Validation auc = 0.715493
Epoch 161
Validation auc = 0.723944
Epoch 162
Validation auc = 0.729108
Epoch 163
Validation auc = 0.730986
Epoch 164
Loss = 4.9270e-03, PNorm = 109.6320, GNorm = 0.5045, lr_0 = 7.6436e-04
Validation auc = 0.730986
Epoch 165
Validation auc = 0.730986
Epoch 166
Validation auc = 0.730516
Epoch 167
Validation auc = 0.711268
Epoch 168
Validation auc = 0.714554
Epoch 169
Loss = 2.9292e-03, PNorm = 109.6925, GNorm = 0.0995, lr_0 = 7.5782e-04
Validation auc = 0.713146
Epoch 170
Validation auc = 0.712207
Epoch 171
Validation auc = 0.715962
Epoch 172
Validation auc = 0.715023
Epoch 173
Validation auc = 0.716432
Epoch 174
Loss = 1.9426e-03, PNorm = 109.7851, GNorm = 0.0155, lr_0 = 7.5134e-04
Validation auc = 0.700939
Epoch 175
Validation auc = 0.697653
Epoch 176
Validation auc = 0.698592
Epoch 177
Validation auc = 0.700469
Epoch 178
Validation auc = 0.695775
Epoch 179
Loss = 4.2268e-04, PNorm = 109.8794, GNorm = 0.0461, lr_0 = 7.4492e-04
Validation auc = 0.690141
Epoch 180
Validation auc = 0.703286
Epoch 181
Validation auc = 0.719249
Epoch 182
Validation auc = 0.721127
Epoch 183
Validation auc = 0.712207
Epoch 184
Loss = 1.1486e-03, PNorm = 110.0140, GNorm = 0.0195, lr_0 = 7.3855e-04
Validation auc = 0.702347
Epoch 185
Validation auc = 0.699061
Epoch 186
Validation auc = 0.691549
Epoch 187
Validation auc = 0.690141
Epoch 188
Validation auc = 0.684977
Epoch 189
Loss = 8.3214e-02, PNorm = 110.1713, GNorm = 4.0638, lr_0 = 7.3224e-04
Validation auc = 0.705164
Epoch 190
Validation auc = 0.727700
Epoch 191
Validation auc = 0.745540
Epoch 192
Validation auc = 0.731925
Epoch 193
Validation auc = 0.719718
Epoch 194
Loss = 4.4202e-03, PNorm = 110.4045, GNorm = 0.7886, lr_0 = 7.2598e-04
Validation auc = 0.715962
Epoch 195
Validation auc = 0.701878
Epoch 196
Validation auc = 0.687324
Epoch 197
Validation auc = 0.715962
Epoch 198
Validation auc = 0.727230
Epoch 199
Loss = 2.0784e-03, PNorm = 110.6566, GNorm = 0.1333, lr_0 = 7.1977e-04
Validation auc = 0.728169
Epoch 200
Validation auc = 0.724413
Epoch 201
Validation auc = 0.719249
Epoch 202
Validation auc = 0.717840
Epoch 203
Validation auc = 0.715023
Epoch 204
Loss = 8.7285e-04, PNorm = 110.8510, GNorm = 0.0429, lr_0 = 7.1362e-04
Validation auc = 0.721596
Epoch 205
Validation auc = 0.723474
Epoch 206
Validation auc = 0.724413
Epoch 207
Validation auc = 0.724883
Epoch 208
Validation auc = 0.731925
Epoch 209
Loss = 6.7640e-03, PNorm = 110.9574, GNorm = 1.2130, lr_0 = 7.0752e-04
Validation auc = 0.740376
Epoch 210
Validation auc = 0.745540
Epoch 211
Validation auc = 0.756338
Epoch 212
Validation auc = 0.762911
Epoch 213
Validation auc = 0.766667
Epoch 214
Loss = 2.0293e-03, PNorm = 111.0335, GNorm = 0.0347, lr_0 = 7.0147e-04
Validation auc = 0.768545
Epoch 215
Validation auc = 0.770423
Epoch 216
Validation auc = 0.774178
Epoch 217
Validation auc = 0.776526
Epoch 218
Validation auc = 0.771362
Epoch 219
Loss = 2.3561e-04, PNorm = 111.1050, GNorm = 0.0297, lr_0 = 6.9548e-04
Validation auc = 0.772300
Epoch 220
Validation auc = 0.765258
Epoch 221
Validation auc = 0.757042
Epoch 222
Validation auc = 0.752113
Epoch 223
Validation auc = 0.746948
Epoch 224
Loss = 3.8198e-04, PNorm = 111.1727, GNorm = 0.0084, lr_0 = 6.8953e-04
Validation auc = 0.744601
Epoch 225
Validation auc = 0.751643
Epoch 226
Validation auc = 0.755869
Epoch 227
Validation auc = 0.756808
Epoch 228
Validation auc = 0.755399
Epoch 229
Loss = 1.2432e-02, PNorm = 111.2299, GNorm = 1.9450, lr_0 = 6.8364e-04
Validation auc = 0.752582
Epoch 230
Validation auc = 0.750235
Epoch 231
Validation auc = 0.759155
Epoch 232
Validation auc = 0.765728
Epoch 233
Validation auc = 0.770423
Epoch 234
Loss = 1.0737e-02, PNorm = 111.3498, GNorm = 1.2087, lr_0 = 6.7779e-04
Validation auc = 0.770423
Epoch 235
Validation auc = 0.767606
Epoch 236
Validation auc = 0.762441
Epoch 237
Validation auc = 0.758216
Epoch 238
Validation auc = 0.768545
Epoch 239
Loss = 8.0155e-03, PNorm = 111.4957, GNorm = 1.9826, lr_0 = 6.7200e-04
Validation auc = 0.771831
Epoch 240
Validation auc = 0.769953
Epoch 241
Validation auc = 0.766667
Epoch 242
Validation auc = 0.766667
Epoch 243
Validation auc = 0.762441
Epoch 244
Loss = 2.2001e-02, PNorm = 111.6519, GNorm = 0.6626, lr_0 = 6.6625e-04
Validation auc = 0.766197
Epoch 245
Validation auc = 0.768075
Epoch 246
Validation auc = 0.772300
Epoch 247
Validation auc = 0.775587
Epoch 248
Validation auc = 0.773239
Epoch 249
Loss = 1.0665e-02, PNorm = 111.7720, GNorm = 0.4794, lr_0 = 6.6056e-04
Validation auc = 0.760563
Epoch 250
Validation auc = 0.753991
Epoch 251
Validation auc = 0.751174
Epoch 252
Validation auc = 0.741784
Epoch 253
Validation auc = 0.740376
Epoch 254
Loss = 2.5107e-03, PNorm = 111.8594, GNorm = 0.6744, lr_0 = 6.5491e-04
Validation auc = 0.737559
Epoch 255
Validation auc = 0.736620
Epoch 256
Validation auc = 0.733333
Epoch 257
Validation auc = 0.733333
Epoch 258
Validation auc = 0.731455
Epoch 259
Loss = 3.2338e-03, PNorm = 111.9140, GNorm = 0.5907, lr_0 = 6.4931e-04
Validation auc = 0.729108
Epoch 260
Validation auc = 0.730047
Epoch 261
Validation auc = 0.728638
Epoch 262
Validation auc = 0.727700
Epoch 263
Validation auc = 0.726761
Epoch 264
Loss = 3.5619e-04, PNorm = 111.9435, GNorm = 0.0905, lr_0 = 6.4376e-04
Validation auc = 0.726291
Epoch 265
Validation auc = 0.737559
Epoch 266
Validation auc = 0.736150
Epoch 267
Validation auc = 0.736620
Epoch 268
Validation auc = 0.735211
Epoch 269
Loss = 1.2975e-03, PNorm = 112.0180, GNorm = 0.2477, lr_0 = 6.3826e-04
Validation auc = 0.734272
Epoch 270
Validation auc = 0.730986
Epoch 271
Validation auc = 0.732394
Epoch 272
Validation auc = 0.732394
Epoch 273
Validation auc = 0.731455
Epoch 274
Loss = 7.4330e-05, PNorm = 112.0684, GNorm = 0.0039, lr_0 = 6.3280e-04
Validation auc = 0.733333
Epoch 275
Validation auc = 0.738498
Epoch 276
Validation auc = 0.736620
Epoch 277
Validation auc = 0.734272
Epoch 278
Validation auc = 0.733803
Epoch 279
Loss = 1.7662e-02, PNorm = 112.0942, GNorm = 3.6290, lr_0 = 6.2739e-04
Validation auc = 0.740376
Epoch 280
Validation auc = 0.742723
Epoch 281
Validation auc = 0.739906
Epoch 282
Validation auc = 0.730047
Epoch 283
Validation auc = 0.720657
Epoch 284
Loss = 2.4857e-02, PNorm = 112.1791, GNorm = 1.2707, lr_0 = 6.2203e-04
Validation auc = 0.720188
Epoch 285
Validation auc = 0.706103
Epoch 286
Validation auc = 0.713146
Epoch 287
Validation auc = 0.715962
Epoch 288
Validation auc = 0.707042
Epoch 289
Loss = 1.5018e-02, PNorm = 112.3231, GNorm = 1.6789, lr_0 = 6.1671e-04
Validation auc = 0.705164
Epoch 290
Validation auc = 0.710798
Epoch 291
Validation auc = 0.720657
Epoch 292
Validation auc = 0.726761
Epoch 293
Validation auc = 0.717371
Epoch 294
Loss = 6.8695e-03, PNorm = 112.4649, GNorm = 0.4060, lr_0 = 6.1144e-04
Validation auc = 0.697183
Epoch 295
Validation auc = 0.682629
Epoch 296
Validation auc = 0.707512
Epoch 297
Validation auc = 0.713615
Epoch 298
Validation auc = 0.721596
Epoch 299
Loss = 1.2860e-02, PNorm = 112.6026, GNorm = 2.5404, lr_0 = 6.0621e-04
Validation auc = 0.717840
Model 0 best validation auc = 0.776526 on epoch 217
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.907285
Ensemble test auc = 0.907285
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.670423
Epoch 1
Validation auc = 0.653052
Epoch 2
Validation auc = 0.676526
Epoch 3
Validation auc = 0.700469
Epoch 4
Loss = 5.5404e-01, PNorm = 100.5496, GNorm = 2.3688, lr_0 = 6.5000e-04
Validation auc = 0.688732
Epoch 5
Validation auc = 0.676526
Epoch 6
Validation auc = 0.657277
Epoch 7
Validation auc = 0.676995
Epoch 8
Validation auc = 0.654930
Epoch 9
Loss = 3.9781e-01, PNorm = 100.9156, GNorm = 2.5239, lr_0 = 9.9743e-04
Validation auc = 0.659624
Epoch 10
Validation auc = 0.632864
Epoch 11
Validation auc = 0.638498
Epoch 12
Validation auc = 0.656338
Epoch 13
Validation auc = 0.608920
Epoch 14
Loss = 3.7886e-01, PNorm = 101.2963, GNorm = 3.7079, lr_0 = 9.8890e-04
Validation auc = 0.640845
Epoch 15
Validation auc = 0.652582
Epoch 16
Validation auc = 0.661502
Epoch 17
Validation auc = 0.669953
Epoch 18
Validation auc = 0.660563
Epoch 19
Loss = 2.8185e-01, PNorm = 101.6307, GNorm = 2.1431, lr_0 = 9.8045e-04
Validation auc = 0.656338
Epoch 20
Validation auc = 0.644131
Epoch 21
Validation auc = 0.642254
Epoch 22
Validation auc = 0.669014
Epoch 23
Validation auc = 0.691549
Epoch 24
Loss = 3.3657e-01, PNorm = 101.9486, GNorm = 5.6769, lr_0 = 9.7207e-04
Validation auc = 0.690610
Epoch 25
Validation auc = 0.686385
Epoch 26
Validation auc = 0.684038
Epoch 27
Validation auc = 0.649296
Epoch 28
Validation auc = 0.698122
Epoch 29
Loss = 2.3547e-01, PNorm = 102.2378, GNorm = 1.5545, lr_0 = 9.6376e-04
Validation auc = 0.703756
Epoch 30
Validation auc = 0.700939
Epoch 31
Validation auc = 0.683568
Epoch 32
Validation auc = 0.682629
Epoch 33
Validation auc = 0.681221
Epoch 34
Loss = 4.0348e-01, PNorm = 102.5525, GNorm = 6.3725, lr_0 = 9.5552e-04
Validation auc = 0.677934
Epoch 35
Validation auc = 0.676526
Epoch 36
Validation auc = 0.676995
Epoch 37
Validation auc = 0.675117
Epoch 38
Validation auc = 0.689671
Epoch 39
Loss = 1.7253e-01, PNorm = 102.9293, GNorm = 1.4061, lr_0 = 9.4735e-04
Validation auc = 0.694366
Epoch 40
Validation auc = 0.686854
Epoch 41
Validation auc = 0.676526
Epoch 42
Validation auc = 0.661033
Epoch 43
Validation auc = 0.657277
Epoch 44
Loss = 9.6899e-02, PNorm = 103.2751, GNorm = 2.8640, lr_0 = 9.3925e-04
Validation auc = 0.654460
Epoch 45
Validation auc = 0.651174
Epoch 46
Validation auc = 0.652113
Epoch 47
Validation auc = 0.667606
Epoch 48
Validation auc = 0.685915
Epoch 49
Loss = 1.9655e-01, PNorm = 103.6137, GNorm = 7.0054, lr_0 = 9.3122e-04
Validation auc = 0.681221
Epoch 50
Validation auc = 0.669953
Epoch 51
Validation auc = 0.674648
Epoch 52
Validation auc = 0.676526
Epoch 53
Validation auc = 0.681221
Epoch 54
Loss = 1.0143e-01, PNorm = 103.9682, GNorm = 1.8257, lr_0 = 9.2326e-04
Validation auc = 0.688732
Epoch 55
Validation auc = 0.689202
Epoch 56
Validation auc = 0.685915
Epoch 57
Validation auc = 0.686385
Epoch 58
Validation auc = 0.685915
Epoch 59
Loss = 1.3274e-01, PNorm = 104.2915, GNorm = 2.5026, lr_0 = 9.1537e-04
Validation auc = 0.677934
Epoch 60
Validation auc = 0.663850
Epoch 61
Validation auc = 0.669014
Epoch 62
Validation auc = 0.669484
Epoch 63
Validation auc = 0.657277
Epoch 64
Loss = 2.1115e-01, PNorm = 104.6354, GNorm = 7.2956, lr_0 = 9.0754e-04
Validation auc = 0.621596
Epoch 65
Validation auc = 0.615493
Epoch 66
Validation auc = 0.629577
Epoch 67
Validation auc = 0.687793
Epoch 68
Validation auc = 0.696244
Epoch 69
Loss = 9.7443e-02, PNorm = 105.0167, GNorm = 4.0715, lr_0 = 8.9978e-04
Validation auc = 0.703286
Epoch 70
Validation auc = 0.699061
Epoch 71
Validation auc = 0.678404
Epoch 72
Validation auc = 0.693897
Epoch 73
Validation auc = 0.707512
Epoch 74
Loss = 1.3793e-01, PNorm = 105.4845, GNorm = 2.2894, lr_0 = 8.9209e-04
Validation auc = 0.702817
Epoch 75
Validation auc = 0.693897
Epoch 76
Validation auc = 0.682629
Epoch 77
Validation auc = 0.681221
Epoch 78
Validation auc = 0.682160
Epoch 79
Loss = 1.4023e-01, PNorm = 105.9598, GNorm = 2.9677, lr_0 = 8.8447e-04
Validation auc = 0.694366
Epoch 80
Validation auc = 0.702347
Epoch 81
Validation auc = 0.706103
Epoch 82
Validation auc = 0.701878
Epoch 83
Validation auc = 0.690141
Epoch 84
Loss = 6.8406e-02, PNorm = 106.3114, GNorm = 2.9741, lr_0 = 8.7691e-04
Validation auc = 0.685446
Epoch 85
Validation auc = 0.690610
Epoch 86
Validation auc = 0.683568
Epoch 87
Validation auc = 0.678873
Epoch 88
Validation auc = 0.673709
Epoch 89
Loss = 3.2029e-02, PNorm = 106.5822, GNorm = 3.0479, lr_0 = 8.6941e-04
Validation auc = 0.677465
Epoch 90
Validation auc = 0.684038
Epoch 91
Validation auc = 0.691080
Epoch 92
Validation auc = 0.704225
Epoch 93
Validation auc = 0.707512
Epoch 94
Loss = 9.1131e-02, PNorm = 106.8072, GNorm = 4.5901, lr_0 = 8.6198e-04
Validation auc = 0.697183
Epoch 95
Validation auc = 0.694366
Epoch 96
Validation auc = 0.691080
Epoch 97
Validation auc = 0.694366
Epoch 98
Validation auc = 0.679812
Epoch 99
Loss = 3.9489e-02, PNorm = 107.0607, GNorm = 1.3623, lr_0 = 8.5461e-04
Validation auc = 0.687324
Epoch 100
Validation auc = 0.689202
Epoch 101
Validation auc = 0.676526
Epoch 102
Validation auc = 0.680282
Epoch 103
Validation auc = 0.680282
Epoch 104
Loss = 5.6201e-02, PNorm = 107.3132, GNorm = 2.9991, lr_0 = 8.4730e-04
Validation auc = 0.681221
Epoch 105
Validation auc = 0.672770
Epoch 106
Validation auc = 0.671362
Epoch 107
Validation auc = 0.664789
Epoch 108
Validation auc = 0.661502
Epoch 109
Loss = 6.1727e-02, PNorm = 107.5533, GNorm = 0.9586, lr_0 = 8.4006e-04
Validation auc = 0.666197
Epoch 110
Validation auc = 0.670423
Epoch 111
Validation auc = 0.683568
Epoch 112
Validation auc = 0.683568
Epoch 113
Validation auc = 0.688732
Epoch 114
Loss = 3.3234e-02, PNorm = 107.7673, GNorm = 3.1758, lr_0 = 8.3288e-04
Validation auc = 0.691080
Epoch 115
Validation auc = 0.688732
Epoch 116
Validation auc = 0.694836
Epoch 117
Validation auc = 0.702347
Epoch 118
Validation auc = 0.704695
Epoch 119
Loss = 6.1410e-02, PNorm = 107.9770, GNorm = 0.1222, lr_0 = 8.2576e-04
Validation auc = 0.693427
Epoch 120
Validation auc = 0.680751
Epoch 121
Validation auc = 0.678873
Epoch 122
Validation auc = 0.684507
Epoch 123
Validation auc = 0.679812
Epoch 124
Loss = 1.4456e-02, PNorm = 108.2140, GNorm = 1.6042, lr_0 = 8.1870e-04
Validation auc = 0.676056
Epoch 125
Validation auc = 0.676526
Epoch 126
Validation auc = 0.681690
Epoch 127
Validation auc = 0.693427
Epoch 128
Validation auc = 0.686385
Epoch 129
Loss = 1.1565e-01, PNorm = 108.4664, GNorm = 8.3445, lr_0 = 8.1170e-04
Validation auc = 0.687793
Epoch 130
Validation auc = 0.707042
Epoch 131
Validation auc = 0.725822
Epoch 132
Validation auc = 0.736620
Epoch 133
Validation auc = 0.726761
Epoch 134
Loss = 5.3235e-02, PNorm = 108.8777, GNorm = 0.7381, lr_0 = 8.0476e-04
Validation auc = 0.710329
Epoch 135
Validation auc = 0.700939
Epoch 136
Validation auc = 0.705164
Epoch 137
Validation auc = 0.699061
Epoch 138
Validation auc = 0.691549
Epoch 139
Loss = 4.2211e-02, PNorm = 109.1721, GNorm = 2.3967, lr_0 = 7.9788e-04
Validation auc = 0.695775
Epoch 140
Validation auc = 0.703756
Epoch 141
Validation auc = 0.708920
Epoch 142
Validation auc = 0.715962
Epoch 143
Validation auc = 0.718779
Epoch 144
Loss = 2.6458e-02, PNorm = 109.3752, GNorm = 1.1298, lr_0 = 7.9106e-04
Validation auc = 0.715962
Epoch 145
Validation auc = 0.716432
Epoch 146
Validation auc = 0.705634
Epoch 147
Validation auc = 0.706573
Epoch 148
Validation auc = 0.714554
Epoch 149
Loss = 8.2939e-03, PNorm = 109.5671, GNorm = 0.5563, lr_0 = 7.8430e-04
Validation auc = 0.717371
Epoch 150
Validation auc = 0.718779
Epoch 151
Validation auc = 0.723474
Epoch 152
Validation auc = 0.724413
Epoch 153
Validation auc = 0.725822
Epoch 154
Loss = 9.0649e-03, PNorm = 109.7104, GNorm = 0.0594, lr_0 = 7.7759e-04
Validation auc = 0.727230
Epoch 155
Validation auc = 0.730516
Epoch 156
Validation auc = 0.721596
Epoch 157
Validation auc = 0.709859
Epoch 158
Validation auc = 0.701878
Epoch 159
Loss = 1.0995e-02, PNorm = 109.8299, GNorm = 0.4537, lr_0 = 7.7095e-04
Validation auc = 0.693897
Epoch 160
Validation auc = 0.681690
Epoch 161
Validation auc = 0.672770
Epoch 162
Validation auc = 0.678404
Epoch 163
Validation auc = 0.688263
Epoch 164
Loss = 7.8680e-03, PNorm = 109.9782, GNorm = 0.8769, lr_0 = 7.6436e-04
Validation auc = 0.691080
Epoch 165
Validation auc = 0.692019
Epoch 166
Validation auc = 0.692488
Epoch 167
Validation auc = 0.690610
Epoch 168
Validation auc = 0.692958
Epoch 169
Loss = 1.3190e-02, PNorm = 110.0984, GNorm = 1.4868, lr_0 = 7.5782e-04
Validation auc = 0.700469
Epoch 170
Validation auc = 0.706103
Epoch 171
Validation auc = 0.707042
Epoch 172
Validation auc = 0.709859
Epoch 173
Validation auc = 0.710329
Epoch 174
Loss = 5.2137e-04, PNorm = 110.1978, GNorm = 0.0280, lr_0 = 7.5134e-04
Validation auc = 0.709390
Epoch 175
Validation auc = 0.715023
Epoch 176
Validation auc = 0.716432
Epoch 177
Validation auc = 0.715493
Epoch 178
Validation auc = 0.713615
Epoch 179
Loss = 1.3620e-02, PNorm = 110.2719, GNorm = 0.0755, lr_0 = 7.4492e-04
Validation auc = 0.714554
Epoch 180
Validation auc = 0.718779
Epoch 181
Validation auc = 0.719249
Epoch 182
Validation auc = 0.713146
Epoch 183
Validation auc = 0.709390
Epoch 184
Loss = 3.8017e-03, PNorm = 110.3526, GNorm = 0.8681, lr_0 = 7.3855e-04
Validation auc = 0.704225
Epoch 185
Validation auc = 0.704225
Epoch 186
Validation auc = 0.703756
Epoch 187
Validation auc = 0.705634
Epoch 188
Validation auc = 0.709390
Epoch 189
Loss = 2.1724e-03, PNorm = 110.4252, GNorm = 0.4073, lr_0 = 7.3224e-04
Validation auc = 0.707042
Epoch 190
Validation auc = 0.703756
Epoch 191
Validation auc = 0.706103
Epoch 192
Validation auc = 0.706573
Epoch 193
Validation auc = 0.709390
Epoch 194
Loss = 1.0654e-03, PNorm = 110.4715, GNorm = 0.0635, lr_0 = 7.2598e-04
Validation auc = 0.708451
Epoch 195
Validation auc = 0.709859
Epoch 196
Validation auc = 0.710329
Epoch 197
Validation auc = 0.711737
Epoch 198
Validation auc = 0.713146
Epoch 199
Loss = 4.7702e-04, PNorm = 110.5033, GNorm = 0.0164, lr_0 = 7.1977e-04
Validation auc = 0.710329
Epoch 200
Validation auc = 0.711737
Epoch 201
Validation auc = 0.710798
Epoch 202
Validation auc = 0.710798
Epoch 203
Validation auc = 0.710798
Epoch 204
Loss = 1.6551e-03, PNorm = 110.5243, GNorm = 0.0095, lr_0 = 7.1362e-04
Validation auc = 0.708920
Epoch 205
Validation auc = 0.707042
Epoch 206
Validation auc = 0.707981
Epoch 207
Validation auc = 0.708920
Epoch 208
Validation auc = 0.707981
Epoch 209
Loss = 1.6603e-02, PNorm = 110.5398, GNorm = 0.0264, lr_0 = 7.0752e-04
Validation auc = 0.705634
Epoch 210
Validation auc = 0.703756
Epoch 211
Validation auc = 0.707981
Epoch 212
Validation auc = 0.709859
Epoch 213
Validation auc = 0.709390
Epoch 214
Loss = 1.3785e-02, PNorm = 110.5579, GNorm = 3.1240, lr_0 = 7.0147e-04
Validation auc = 0.706103
Epoch 215
Validation auc = 0.706573
Epoch 216
Validation auc = 0.707512
Epoch 217
Validation auc = 0.705164
Epoch 218
Validation auc = 0.707042
Epoch 219
Loss = 4.0336e-02, PNorm = 110.5886, GNorm = 4.0915, lr_0 = 6.9548e-04
Validation auc = 0.710329
Epoch 220
Validation auc = 0.717840
Epoch 221
Validation auc = 0.728638
Epoch 222
Validation auc = 0.737559
Epoch 223
Validation auc = 0.743192
Epoch 224
Loss = 1.0449e-02, PNorm = 110.6675, GNorm = 2.1748, lr_0 = 6.8953e-04
Validation auc = 0.738028
Epoch 225
Validation auc = 0.730516
Epoch 226
Validation auc = 0.736150
Epoch 227
Validation auc = 0.737559
Epoch 228
Validation auc = 0.731455
Epoch 229
Loss = 5.7098e-03, PNorm = 110.7986, GNorm = 0.1504, lr_0 = 6.8364e-04
Validation auc = 0.729577
Epoch 230
Validation auc = 0.720657
Epoch 231
Validation auc = 0.719249
Epoch 232
Validation auc = 0.718779
Epoch 233
Validation auc = 0.720188
Epoch 234
Loss = 2.2814e-03, PNorm = 110.9020, GNorm = 0.2074, lr_0 = 6.7779e-04
Validation auc = 0.717840
Epoch 235
Validation auc = 0.709390
Epoch 236
Validation auc = 0.714554
Epoch 237
Validation auc = 0.723474
Epoch 238
Validation auc = 0.732864
Epoch 239
Loss = 5.0020e-02, PNorm = 111.0507, GNorm = 3.2149, lr_0 = 6.7200e-04
Validation auc = 0.743662
Epoch 240
Validation auc = 0.742723
Epoch 241
Validation auc = 0.740845
Epoch 242
Validation auc = 0.744131
Epoch 243
Validation auc = 0.748826
Epoch 244
Loss = 2.0199e-02, PNorm = 111.2437, GNorm = 2.6911, lr_0 = 6.6625e-04
Validation auc = 0.744601
Epoch 245
Validation auc = 0.737559
Epoch 246
Validation auc = 0.728638
Epoch 247
Validation auc = 0.699061
Epoch 248
Validation auc = 0.674648
Epoch 249
Loss = 4.6619e-03, PNorm = 111.4587, GNorm = 0.7036, lr_0 = 6.6056e-04
Validation auc = 0.662911
Epoch 250
Validation auc = 0.646948
Epoch 251
Validation auc = 0.698592
Epoch 252
Validation auc = 0.696714
Epoch 253
Validation auc = 0.702817
Epoch 254
Loss = 1.1796e-01, PNorm = 111.7653, GNorm = 5.2830, lr_0 = 6.5491e-04
Validation auc = 0.712676
Epoch 255
Validation auc = 0.704225
Epoch 256
Validation auc = 0.704695
Epoch 257
Validation auc = 0.707981
Epoch 258
Validation auc = 0.702347
Epoch 259
Loss = 1.9010e-01, PNorm = 112.2128, GNorm = 0.4539, lr_0 = 6.4931e-04
Validation auc = 0.708920
Epoch 260
Validation auc = 0.709859
Epoch 261
Validation auc = 0.723474
Epoch 262
Validation auc = 0.723944
Epoch 263
Validation auc = 0.720657
Epoch 264
Loss = 8.0053e-02, PNorm = 112.5359, GNorm = 1.3386, lr_0 = 6.4376e-04
Validation auc = 0.726291
Epoch 265
Validation auc = 0.722066
Epoch 266
Validation auc = 0.717371
Epoch 267
Validation auc = 0.705164
Epoch 268
Validation auc = 0.693427
Epoch 269
Loss = 2.1393e-02, PNorm = 112.7808, GNorm = 1.0243, lr_0 = 6.3826e-04
Validation auc = 0.683099
Epoch 270
Validation auc = 0.694366
Epoch 271
Validation auc = 0.696714
Epoch 272
Validation auc = 0.690141
Epoch 273
Validation auc = 0.688732
Epoch 274
Loss = 2.4015e-03, PNorm = 112.9595, GNorm = 0.0650, lr_0 = 6.3280e-04
Validation auc = 0.683568
Epoch 275
Validation auc = 0.677934
Epoch 276
Validation auc = 0.676526
Epoch 277
Validation auc = 0.682629
Epoch 278
Validation auc = 0.687324
Epoch 279
Loss = 1.0807e-02, PNorm = 113.1028, GNorm = 0.4558, lr_0 = 6.2739e-04
Validation auc = 0.690610
Epoch 280
Validation auc = 0.692488
Epoch 281
Validation auc = 0.694366
Epoch 282
Validation auc = 0.699531
Epoch 283
Validation auc = 0.703286
Epoch 284
Loss = 6.4356e-04, PNorm = 113.2063, GNorm = 0.0168, lr_0 = 6.2203e-04
Validation auc = 0.701408
Epoch 285
Validation auc = 0.704225
Epoch 286
Validation auc = 0.704225
Epoch 287
Validation auc = 0.705164
Epoch 288
Validation auc = 0.707042
Epoch 289
Loss = 1.1971e-03, PNorm = 113.2645, GNorm = 0.0440, lr_0 = 6.1671e-04
Validation auc = 0.708920
Epoch 290
Validation auc = 0.709390
Epoch 291
Validation auc = 0.708451
Epoch 292
Validation auc = 0.706103
Epoch 293
Validation auc = 0.709390
Epoch 294
Loss = 1.0939e-03, PNorm = 113.3287, GNorm = 0.0578, lr_0 = 6.1144e-04
Validation auc = 0.712676
Epoch 295
Validation auc = 0.715493
Epoch 296
Validation auc = 0.717371
Epoch 297
Validation auc = 0.721596
Epoch 298
Validation auc = 0.721127
Epoch 299
Loss = 4.1003e-03, PNorm = 113.4028, GNorm = 0.4385, lr_0 = 6.0621e-04
Validation auc = 0.721596
Model 0 best validation auc = 0.748826 on epoch 243
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.866948
Ensemble test auc = 0.866948
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.691080
Epoch 1
Validation auc = 0.638028
Epoch 2
Validation auc = 0.682629
Epoch 3
Validation auc = 0.644601
Epoch 4
Loss = 6.4025e-01, PNorm = 100.5455, GNorm = 9.6638, lr_0 = 6.5000e-04
Validation auc = 0.593897
Epoch 5
Validation auc = 0.671362
Epoch 6
Validation auc = 0.638498
Epoch 7
Validation auc = 0.693897
Epoch 8
Validation auc = 0.700939
Epoch 9
Loss = 5.5828e-01, PNorm = 100.9033, GNorm = 2.1045, lr_0 = 9.9743e-04
Validation auc = 0.675587
Epoch 10
Validation auc = 0.688263
Epoch 11
Validation auc = 0.702347
Epoch 12
Validation auc = 0.710329
Epoch 13
Validation auc = 0.698592
Epoch 14
Loss = 4.8538e-01, PNorm = 101.2677, GNorm = 7.1042, lr_0 = 9.8890e-04
Validation auc = 0.693897
Epoch 15
Validation auc = 0.688263
Epoch 16
Validation auc = 0.682629
Epoch 17
Validation auc = 0.674648
Epoch 18
Validation auc = 0.680751
Epoch 19
Loss = 3.9450e-01, PNorm = 101.5694, GNorm = 3.5551, lr_0 = 9.8045e-04
Validation auc = 0.679812
Epoch 20
Validation auc = 0.671362
Epoch 21
Validation auc = 0.660563
Epoch 22
Validation auc = 0.648357
Epoch 23
Validation auc = 0.658216
Epoch 24
Loss = 3.1509e-01, PNorm = 101.8436, GNorm = 1.9124, lr_0 = 9.7207e-04
Validation auc = 0.682160
Epoch 25
Validation auc = 0.674648
Epoch 26
Validation auc = 0.681221
Epoch 27
Validation auc = 0.681221
Epoch 28
Validation auc = 0.672300
Epoch 29
Loss = 4.0194e-01, PNorm = 102.1447, GNorm = 7.8839, lr_0 = 9.6376e-04
Validation auc = 0.674648
Epoch 30
Validation auc = 0.698122
Epoch 31
Validation auc = 0.683568
Epoch 32
Validation auc = 0.631925
Epoch 33
Validation auc = 0.660094
Epoch 34
Loss = 3.6876e-01, PNorm = 102.4572, GNorm = 5.7257, lr_0 = 9.5552e-04
Validation auc = 0.686385
Epoch 35
Validation auc = 0.694366
Epoch 36
Validation auc = 0.710329
Epoch 37
Validation auc = 0.722066
Epoch 38
Validation auc = 0.715962
Epoch 39
Loss = 2.6029e-01, PNorm = 102.8055, GNorm = 5.0610, lr_0 = 9.4735e-04
Validation auc = 0.714554
Epoch 40
Validation auc = 0.714554
Epoch 41
Validation auc = 0.686385
Epoch 42
Validation auc = 0.680282
Epoch 43
Validation auc = 0.679343
Epoch 44
Loss = 2.1909e-01, PNorm = 103.1596, GNorm = 2.7266, lr_0 = 9.3925e-04
Validation auc = 0.678873
Epoch 45
Validation auc = 0.675117
Epoch 46
Validation auc = 0.677934
Epoch 47
Validation auc = 0.692488
Epoch 48
Validation auc = 0.683568
Epoch 49
Loss = 1.7522e-01, PNorm = 103.5649, GNorm = 8.7982, lr_0 = 9.3122e-04
Validation auc = 0.684507
Epoch 50
Validation auc = 0.712676
Epoch 51
Validation auc = 0.715493
Epoch 52
Validation auc = 0.706573
Epoch 53
Validation auc = 0.702347
Epoch 54
Loss = 1.6777e-01, PNorm = 103.9951, GNorm = 2.1321, lr_0 = 9.2326e-04
Validation auc = 0.698592
Epoch 55
Validation auc = 0.703286
Epoch 56
Validation auc = 0.704225
Epoch 57
Validation auc = 0.716432
Epoch 58
Validation auc = 0.714554
Epoch 59
Loss = 8.1912e-02, PNorm = 104.4052, GNorm = 1.5998, lr_0 = 9.1537e-04
Validation auc = 0.710329
Epoch 60
Validation auc = 0.711737
Epoch 61
Validation auc = 0.694836
Epoch 62
Validation auc = 0.700469
Epoch 63
Validation auc = 0.727700
Epoch 64
Loss = 6.3495e-02, PNorm = 104.7879, GNorm = 1.6318, lr_0 = 9.0754e-04
Validation auc = 0.742254
Epoch 65
Validation auc = 0.723944
Epoch 66
Validation auc = 0.705634
Epoch 67
Validation auc = 0.680282
Epoch 68
Validation auc = 0.671831
Epoch 69
Loss = 1.0100e-01, PNorm = 105.2372, GNorm = 3.3296, lr_0 = 8.9978e-04
Validation auc = 0.658685
Epoch 70
Validation auc = 0.667606
Epoch 71
Validation auc = 0.674178
Epoch 72
Validation auc = 0.696244
Epoch 73
Validation auc = 0.703286
Epoch 74
Loss = 1.0163e-01, PNorm = 105.6211, GNorm = 3.4076, lr_0 = 8.9209e-04
Validation auc = 0.698592
Epoch 75
Validation auc = 0.707512
Epoch 76
Validation auc = 0.707042
Epoch 77
Validation auc = 0.716432
Epoch 78
Validation auc = 0.722535
Epoch 79
Loss = 2.0865e-02, PNorm = 105.9589, GNorm = 0.1908, lr_0 = 8.8447e-04
Validation auc = 0.738967
Epoch 80
Validation auc = 0.744131
Epoch 81
Validation auc = 0.740376
Epoch 82
Validation auc = 0.731925
Epoch 83
Validation auc = 0.729577
Epoch 84
Loss = 7.2004e-02, PNorm = 106.2556, GNorm = 6.8882, lr_0 = 8.7691e-04
Validation auc = 0.736620
Epoch 85
Validation auc = 0.737089
Epoch 86
Validation auc = 0.737559
Epoch 87
Validation auc = 0.735211
Epoch 88
Validation auc = 0.740845
Epoch 89
Loss = 9.5277e-02, PNorm = 106.6478, GNorm = 2.8422, lr_0 = 8.6941e-04
Validation auc = 0.739906
Epoch 90
Validation auc = 0.738967
Epoch 91
Validation auc = 0.735681
Epoch 92
Validation auc = 0.730516
Epoch 93
Validation auc = 0.734742
Epoch 94
Loss = 9.3005e-02, PNorm = 107.0427, GNorm = 2.5993, lr_0 = 8.6198e-04
Validation auc = 0.733333
Epoch 95
Validation auc = 0.745070
Epoch 96
Validation auc = 0.722066
Epoch 97
Validation auc = 0.694366
Epoch 98
Validation auc = 0.707512
Epoch 99
Loss = 3.3343e-02, PNorm = 107.3696, GNorm = 0.3358, lr_0 = 8.5461e-04
Validation auc = 0.715962
Epoch 100
Validation auc = 0.719249
Epoch 101
Validation auc = 0.726291
Epoch 102
Validation auc = 0.722535
Epoch 103
Validation auc = 0.723944
Epoch 104
Loss = 8.6889e-02, PNorm = 107.6312, GNorm = 2.2354, lr_0 = 8.4730e-04
Validation auc = 0.719249
Epoch 105
Validation auc = 0.715493
Epoch 106
Validation auc = 0.715962
Epoch 107
Validation auc = 0.720188
Epoch 108
Validation auc = 0.722066
Epoch 109
Loss = 2.5434e-02, PNorm = 107.9060, GNorm = 1.5754, lr_0 = 8.4006e-04
Validation auc = 0.724413
Epoch 110
Validation auc = 0.714085
Epoch 111
Validation auc = 0.709859
Epoch 112
Validation auc = 0.706573
Epoch 113
Validation auc = 0.712207
Epoch 114
Loss = 3.4913e-02, PNorm = 108.1107, GNorm = 1.8068, lr_0 = 8.3288e-04
Validation auc = 0.715493
Epoch 115
Validation auc = 0.716432
Epoch 116
Validation auc = 0.711737
Epoch 117
Validation auc = 0.709390
Epoch 118
Validation auc = 0.707981
Epoch 119
Loss = 4.8311e-03, PNorm = 108.3311, GNorm = 0.3657, lr_0 = 8.2576e-04
Validation auc = 0.712207
Epoch 120
Validation auc = 0.711268
Epoch 121
Validation auc = 0.716901
Epoch 122
Validation auc = 0.722535
Epoch 123
Validation auc = 0.736150
Epoch 124
Loss = 1.8527e-02, PNorm = 108.5334, GNorm = 0.1332, lr_0 = 8.1870e-04
Validation auc = 0.747418
Epoch 125
Validation auc = 0.748826
Epoch 126
Validation auc = 0.751643
Epoch 127
Validation auc = 0.750704
Epoch 128
Validation auc = 0.748826
Epoch 129
Loss = 1.6214e-03, PNorm = 108.6712, GNorm = 0.0795, lr_0 = 8.1170e-04
Validation auc = 0.752582
Epoch 130
Validation auc = 0.748826
Epoch 131
Validation auc = 0.743662
Epoch 132
Validation auc = 0.738498
Epoch 133
Validation auc = 0.730516
Epoch 134
Loss = 8.4563e-03, PNorm = 108.7772, GNorm = 1.6367, lr_0 = 8.0476e-04
Validation auc = 0.724883
Epoch 135
Validation auc = 0.724883
Epoch 136
Validation auc = 0.719718
Epoch 137
Validation auc = 0.724883
Epoch 138
Validation auc = 0.733333
Epoch 139
Loss = 4.0636e-03, PNorm = 108.8629, GNorm = 0.2898, lr_0 = 7.9788e-04
Validation auc = 0.742254
Epoch 140
Validation auc = 0.743662
Epoch 141
Validation auc = 0.738498
Epoch 142
Validation auc = 0.742254
Epoch 143
Validation auc = 0.763850
Epoch 144
Loss = 2.3136e-03, PNorm = 108.9585, GNorm = 0.2482, lr_0 = 7.9106e-04
Validation auc = 0.770423
Epoch 145
Validation auc = 0.769014
Epoch 146
Validation auc = 0.752582
Epoch 147
Validation auc = 0.746009
Epoch 148
Validation auc = 0.749765
Epoch 149
Loss = 4.0053e-02, PNorm = 109.1175, GNorm = 0.2457, lr_0 = 7.8430e-04
Validation auc = 0.753052
Epoch 150
Validation auc = 0.749296
Epoch 151
Validation auc = 0.747887
Epoch 152
Validation auc = 0.750704
Epoch 153
Validation auc = 0.749296
Epoch 154
Loss = 3.2345e-02, PNorm = 109.3104, GNorm = 1.8048, lr_0 = 7.7759e-04
Validation auc = 0.733333
Epoch 155
Validation auc = 0.722535
Epoch 156
Validation auc = 0.718779
Epoch 157
Validation auc = 0.706103
Epoch 158
Validation auc = 0.713615
Epoch 159
Loss = 4.6846e-02, PNorm = 109.5638, GNorm = 2.8745, lr_0 = 7.7095e-04
Validation auc = 0.720188
Epoch 160
Validation auc = 0.732394
Epoch 161
Validation auc = 0.744601
Epoch 162
Validation auc = 0.747418
Epoch 163
Validation auc = 0.764789
Epoch 164
Loss = 1.1089e-01, PNorm = 109.7905, GNorm = 1.5212, lr_0 = 7.6436e-04
Validation auc = 0.769484
Epoch 165
Validation auc = 0.761033
Epoch 166
Validation auc = 0.760094
Epoch 167
Validation auc = 0.754930
Epoch 168
Validation auc = 0.751174
Epoch 169
Loss = 2.5546e-02, PNorm = 110.1054, GNorm = 0.9784, lr_0 = 7.5782e-04
Validation auc = 0.744131
Epoch 170
Validation auc = 0.726761
Epoch 171
Validation auc = 0.713146
Epoch 172
Validation auc = 0.702347
Epoch 173
Validation auc = 0.699061
Epoch 174
Loss = 2.1823e-02, PNorm = 110.3371, GNorm = 1.5455, lr_0 = 7.5134e-04
Validation auc = 0.700469
Epoch 175
Validation auc = 0.711268
Epoch 176
Validation auc = 0.719718
Epoch 177
Validation auc = 0.726761
Epoch 178
Validation auc = 0.734272
Epoch 179
Loss = 1.6391e-02, PNorm = 110.4863, GNorm = 0.4936, lr_0 = 7.4492e-04
Validation auc = 0.745540
Epoch 180
Validation auc = 0.746948
Epoch 181
Validation auc = 0.751643
Epoch 182
Validation auc = 0.761033
Epoch 183
Validation auc = 0.776056
Epoch 184
Loss = 1.7566e-03, PNorm = 110.6325, GNorm = 0.0337, lr_0 = 7.3855e-04
Validation auc = 0.785446
Epoch 185
Validation auc = 0.775587
Epoch 186
Validation auc = 0.764319
Epoch 187
Validation auc = 0.761033
Epoch 188
Validation auc = 0.760563
Epoch 189
Loss = 2.6519e-03, PNorm = 110.7523, GNorm = 0.5005, lr_0 = 7.3224e-04
Validation auc = 0.761972
Epoch 190
Validation auc = 0.757277
Epoch 191
Validation auc = 0.746009
Epoch 192
Validation auc = 0.741315
Epoch 193
Validation auc = 0.731925
Epoch 194
Loss = 1.4826e-02, PNorm = 110.8873, GNorm = 1.5402, lr_0 = 7.2598e-04
Validation auc = 0.720188
Epoch 195
Validation auc = 0.711737
Epoch 196
Validation auc = 0.705634
Epoch 197
Validation auc = 0.696244
Epoch 198
Validation auc = 0.696244
Epoch 199
Loss = 7.0371e-03, PNorm = 111.0089, GNorm = 0.3583, lr_0 = 7.1977e-04
Validation auc = 0.696244
Epoch 200
Validation auc = 0.692019
Epoch 201
Validation auc = 0.685446
Epoch 202
Validation auc = 0.688263
Epoch 203
Validation auc = 0.692488
Epoch 204
Loss = 5.7006e-03, PNorm = 111.0963, GNorm = 0.9032, lr_0 = 7.1362e-04
Validation auc = 0.684507
Epoch 205
Validation auc = 0.681690
Epoch 206
Validation auc = 0.667136
Epoch 207
Validation auc = 0.662441
Epoch 208
Validation auc = 0.660094
Epoch 209
Loss = 4.5080e-04, PNorm = 111.2098, GNorm = 0.0471, lr_0 = 7.0752e-04
Validation auc = 0.657277
Epoch 210
Validation auc = 0.657746
Epoch 211
Validation auc = 0.662911
Epoch 212
Validation auc = 0.676056
Epoch 213
Validation auc = 0.702817
Epoch 214
Loss = 3.9002e-02, PNorm = 111.3236, GNorm = 2.7573, lr_0 = 7.0147e-04
Validation auc = 0.727230
Epoch 215
Validation auc = 0.742254
Epoch 216
Validation auc = 0.741315
Epoch 217
Validation auc = 0.736620
Epoch 218
Validation auc = 0.725822
Epoch 219
Loss = 6.0974e-03, PNorm = 111.4989, GNorm = 0.5232, lr_0 = 6.9548e-04
Validation auc = 0.709859
Epoch 220
Validation auc = 0.701878
Epoch 221
Validation auc = 0.701878
Epoch 222
Validation auc = 0.689671
Epoch 223
Validation auc = 0.700469
Epoch 224
Loss = 3.0265e-03, PNorm = 111.6684, GNorm = 0.3763, lr_0 = 6.8953e-04
Validation auc = 0.704225
Epoch 225
Validation auc = 0.698592
Epoch 226
Validation auc = 0.687324
Epoch 227
Validation auc = 0.673709
Epoch 228
Validation auc = 0.672300
Epoch 229
Loss = 2.6924e-02, PNorm = 111.8178, GNorm = 1.0342, lr_0 = 6.8364e-04
Validation auc = 0.680751
Epoch 230
Validation auc = 0.695775
Epoch 231
Validation auc = 0.707512
Epoch 232
Validation auc = 0.715023
Epoch 233
Validation auc = 0.721127
Epoch 234
Loss = 9.0800e-04, PNorm = 111.9600, GNorm = 0.0155, lr_0 = 6.7779e-04
Validation auc = 0.727700
Epoch 235
Validation auc = 0.723944
Epoch 236
Validation auc = 0.725822
Epoch 237
Validation auc = 0.728638
Epoch 238
Validation auc = 0.729108
Epoch 239
Loss = 9.6663e-04, PNorm = 112.0401, GNorm = 0.1139, lr_0 = 6.7200e-04
Validation auc = 0.729577
Epoch 240
Validation auc = 0.717371
Epoch 241
Validation auc = 0.704695
Epoch 242
Validation auc = 0.705164
Epoch 243
Validation auc = 0.689671
Epoch 244
Loss = 1.1196e-03, PNorm = 112.0952, GNorm = 0.0925, lr_0 = 6.6625e-04
Validation auc = 0.680751
Epoch 245
Validation auc = 0.676995
Epoch 246
Validation auc = 0.676526
Epoch 247
Validation auc = 0.674648
Epoch 248
Validation auc = 0.673709
Epoch 249
Loss = 4.2905e-04, PNorm = 112.1459, GNorm = 0.0087, lr_0 = 6.6056e-04
Validation auc = 0.672300
Epoch 250
Validation auc = 0.668075
Epoch 251
Validation auc = 0.668075
Epoch 252
Validation auc = 0.665258
Epoch 253
Validation auc = 0.664789
Epoch 254
Loss = 4.1358e-03, PNorm = 112.1827, GNorm = 0.0963, lr_0 = 6.5491e-04
Validation auc = 0.671362
Epoch 255
Validation auc = 0.677465
Epoch 256
Validation auc = 0.679812
Epoch 257
Validation auc = 0.683099
Epoch 258
Validation auc = 0.686385
Epoch 259
Loss = 7.5835e-05, PNorm = 112.2154, GNorm = 0.0053, lr_0 = 6.4931e-04
Validation auc = 0.688732
Epoch 260
Validation auc = 0.691080
Epoch 261
Validation auc = 0.693897
Epoch 262
Validation auc = 0.695775
Epoch 263
Validation auc = 0.694131
Epoch 264
Loss = 3.1426e-03, PNorm = 112.2319, GNorm = 0.0110, lr_0 = 6.4376e-04
Validation auc = 0.694366
Epoch 265
Validation auc = 0.697418
Epoch 266
Validation auc = 0.698122
Epoch 267
Validation auc = 0.700000
Epoch 268
Validation auc = 0.700939
Epoch 269
Loss = 1.5926e-03, PNorm = 112.2456, GNorm = 0.0451, lr_0 = 6.3826e-04
Validation auc = 0.700469
Epoch 270
Validation auc = 0.700000
Epoch 271
Validation auc = 0.699531
Epoch 272
Validation auc = 0.700000
Epoch 273
Validation auc = 0.699061
Epoch 274
Loss = 1.9714e-04, PNorm = 112.2582, GNorm = 0.0060, lr_0 = 6.3280e-04
Validation auc = 0.698122
Epoch 275
Validation auc = 0.698592
Epoch 276
Validation auc = 0.697653
Epoch 277
Validation auc = 0.696714
Epoch 278
Validation auc = 0.698826
Epoch 279
Loss = 3.1945e-04, PNorm = 112.2696, GNorm = 0.0823, lr_0 = 6.2739e-04
Validation auc = 0.699296
Epoch 280
Validation auc = 0.695070
Epoch 281
Validation auc = 0.696244
Epoch 282
Validation auc = 0.697653
Epoch 283
Validation auc = 0.698592
Epoch 284
Loss = 9.2180e-05, PNorm = 112.2760, GNorm = 0.0092, lr_0 = 6.2203e-04
Validation auc = 0.697653
Epoch 285
Validation auc = 0.696244
Epoch 286
Validation auc = 0.695775
Epoch 287
Validation auc = 0.695775
Epoch 288
Validation auc = 0.696244
Epoch 289
Loss = 1.3805e-04, PNorm = 112.2831, GNorm = 0.0095, lr_0 = 6.1671e-04
Validation auc = 0.696244
Epoch 290
Validation auc = 0.695305
Epoch 291
Validation auc = 0.694836
Epoch 292
Validation auc = 0.694366
Epoch 293
Validation auc = 0.693897
Epoch 294
Loss = 6.2240e-05, PNorm = 112.2897, GNorm = 0.0045, lr_0 = 6.1144e-04
Validation auc = 0.693427
Epoch 295
Validation auc = 0.692958
Epoch 296
Validation auc = 0.697653
Epoch 297
Validation auc = 0.698592
Epoch 298
Validation auc = 0.695775
Epoch 299
Loss = 1.1754e-04, PNorm = 112.2984, GNorm = 0.0106, lr_0 = 6.0621e-04
Validation auc = 0.692958
Model 0 best validation auc = 0.785446 on epoch 184
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.862131
Ensemble test auc = 0.862131
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.663850
Epoch 1
Validation auc = 0.665728
Epoch 2
Validation auc = 0.696244
Epoch 3
Validation auc = 0.679343
Epoch 4
Loss = 5.9124e-01, PNorm = 100.5491, GNorm = 5.0795, lr_0 = 6.5000e-04
Validation auc = 0.614085
Epoch 5
Validation auc = 0.676995
Epoch 6
Validation auc = 0.664319
Epoch 7
Validation auc = 0.677934
Epoch 8
Validation auc = 0.657277
Epoch 9
Loss = 6.2376e-01, PNorm = 100.9036, GNorm = 11.0620, lr_0 = 9.9743e-04
Validation auc = 0.682629
Epoch 10
Validation auc = 0.701408
Epoch 11
Validation auc = 0.696244
Epoch 12
Validation auc = 0.700939
Epoch 13
Validation auc = 0.705634
Epoch 14
Loss = 4.9649e-01, PNorm = 101.2511, GNorm = 2.9383, lr_0 = 9.8890e-04
Validation auc = 0.697653
Epoch 15
Validation auc = 0.700000
Epoch 16
Validation auc = 0.680282
Epoch 17
Validation auc = 0.673239
Epoch 18
Validation auc = 0.683568
Epoch 19
Loss = 3.8482e-01, PNorm = 101.5331, GNorm = 3.4708, lr_0 = 9.8045e-04
Validation auc = 0.687793
Epoch 20
Validation auc = 0.684038
Epoch 21
Validation auc = 0.686854
Epoch 22
Validation auc = 0.678404
Epoch 23
Validation auc = 0.670423
Epoch 24
Loss = 3.6609e-01, PNorm = 101.8188, GNorm = 8.6248, lr_0 = 9.7207e-04
Validation auc = 0.661033
Epoch 25
Validation auc = 0.660563
Epoch 26
Validation auc = 0.662911
Epoch 27
Validation auc = 0.632864
Epoch 28
Validation auc = 0.663850
Epoch 29
Loss = 2.7494e-01, PNorm = 102.0834, GNorm = 4.9182, lr_0 = 9.6376e-04
Validation auc = 0.687793
Epoch 30
Validation auc = 0.694366
Epoch 31
Validation auc = 0.698592
Epoch 32
Validation auc = 0.711268
Epoch 33
Validation auc = 0.726291
Epoch 34
Loss = 2.3276e-01, PNorm = 102.3847, GNorm = 3.1721, lr_0 = 9.5552e-04
Validation auc = 0.704695
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.680751
Epoch 37
Validation auc = 0.690610
Epoch 38
Validation auc = 0.705634
Epoch 39
Loss = 2.6079e-01, PNorm = 102.7351, GNorm = 6.5955, lr_0 = 9.4735e-04
Validation auc = 0.730516
Epoch 40
Validation auc = 0.733333
Epoch 41
Validation auc = 0.694836
Epoch 42
Validation auc = 0.672300
Epoch 43
Validation auc = 0.686854
Epoch 44
Loss = 1.5546e-01, PNorm = 103.1250, GNorm = 4.9280, lr_0 = 9.3925e-04
Validation auc = 0.689202
Epoch 45
Validation auc = 0.688263
Epoch 46
Validation auc = 0.692958
Epoch 47
Validation auc = 0.701878
Epoch 48
Validation auc = 0.714554
Epoch 49
Loss = 1.3238e-01, PNorm = 103.5577, GNorm = 1.3157, lr_0 = 9.3122e-04
Validation auc = 0.723005
Epoch 50
Validation auc = 0.721596
Epoch 51
Validation auc = 0.716432
Epoch 52
Validation auc = 0.701408
Epoch 53
Validation auc = 0.689202
Epoch 54
Loss = 9.2537e-02, PNorm = 104.0039, GNorm = 1.8709, lr_0 = 9.2326e-04
Validation auc = 0.691549
Epoch 55
Validation auc = 0.701878
Epoch 56
Validation auc = 0.709859
Epoch 57
Validation auc = 0.685446
Epoch 58
Validation auc = 0.692488
Epoch 59
Loss = 6.9402e-02, PNorm = 104.4098, GNorm = 0.5940, lr_0 = 9.1537e-04
Validation auc = 0.693897
Epoch 60
Validation auc = 0.694836
Epoch 61
Validation auc = 0.690610
Epoch 62
Validation auc = 0.686854
Epoch 63
Validation auc = 0.683568
Epoch 64
Loss = 1.7551e-01, PNorm = 104.8062, GNorm = 5.0464, lr_0 = 9.0754e-04
Validation auc = 0.678873
Epoch 65
Validation auc = 0.684507
Epoch 66
Validation auc = 0.690845
Epoch 67
Validation auc = 0.694836
Epoch 68
Validation auc = 0.699531
Epoch 69
Loss = 8.2177e-02, PNorm = 105.2098, GNorm = 3.6878, lr_0 = 8.9978e-04
Validation auc = 0.699061
Epoch 70
Validation auc = 0.697183
Epoch 71
Validation auc = 0.716901
Epoch 72
Validation auc = 0.708451
Epoch 73
Validation auc = 0.715962
Epoch 74
Loss = 9.6822e-02, PNorm = 105.6886, GNorm = 1.2369, lr_0 = 8.9209e-04
Validation auc = 0.731455
Epoch 75
Validation auc = 0.731455
Epoch 76
Validation auc = 0.736150
Epoch 77
Validation auc = 0.734272
Epoch 78
Validation auc = 0.714085
Epoch 79
Loss = 2.1594e-02, PNorm = 106.0888, GNorm = 0.6272, lr_0 = 8.8447e-04
Validation auc = 0.704225
Epoch 80
Validation auc = 0.702347
Epoch 81
Validation auc = 0.697183
Epoch 82
Validation auc = 0.726761
Epoch 83
Validation auc = 0.746948
Epoch 84
Loss = 3.8698e-02, PNorm = 106.4108, GNorm = 0.7762, lr_0 = 8.7691e-04
Validation auc = 0.757746
Epoch 85
Validation auc = 0.762911
Epoch 86
Validation auc = 0.764789
Epoch 87
Validation auc = 0.758216
Epoch 88
Validation auc = 0.746948
Epoch 89
Loss = 4.8306e-02, PNorm = 106.7235, GNorm = 2.2637, lr_0 = 8.6941e-04
Validation auc = 0.746009
Epoch 90
Validation auc = 0.749296
Epoch 91
Validation auc = 0.739906
Epoch 92
Validation auc = 0.727230
Epoch 93
Validation auc = 0.706573
Epoch 94
Loss = 8.3721e-02, PNorm = 107.0372, GNorm = 0.8611, lr_0 = 8.6198e-04
Validation auc = 0.729108
Epoch 95
Validation auc = 0.732394
Epoch 96
Validation auc = 0.725822
Epoch 97
Validation auc = 0.725352
Epoch 98
Validation auc = 0.715962
Epoch 99
Loss = 8.7229e-02, PNorm = 107.3542, GNorm = 4.7838, lr_0 = 8.5461e-04
Validation auc = 0.707042
Epoch 100
Validation auc = 0.705634
Epoch 101
Validation auc = 0.668545
Epoch 102
Validation auc = 0.641784
Epoch 103
Validation auc = 0.661033
Epoch 104
Loss = 9.0765e-02, PNorm = 107.6610, GNorm = 6.6389, lr_0 = 8.4730e-04
Validation auc = 0.692488
Epoch 105
Validation auc = 0.698592
Epoch 106
Validation auc = 0.721127
Epoch 107
Validation auc = 0.720657
Epoch 108
Validation auc = 0.720657
Epoch 109
Loss = 3.2254e-02, PNorm = 107.9294, GNorm = 1.2235, lr_0 = 8.4006e-04
Validation auc = 0.728169
Epoch 110
Validation auc = 0.722066
Epoch 111
Validation auc = 0.719718
Epoch 112
Validation auc = 0.716432
Epoch 113
Validation auc = 0.718310
Epoch 114
Loss = 2.8757e-02, PNorm = 108.1862, GNorm = 0.2825, lr_0 = 8.3288e-04
Validation auc = 0.749765
Epoch 115
Validation auc = 0.750235
Epoch 116
Validation auc = 0.743662
Epoch 117
Validation auc = 0.741784
Epoch 118
Validation auc = 0.734272
Epoch 119
Loss = 3.4152e-02, PNorm = 108.4152, GNorm = 0.6729, lr_0 = 8.2576e-04
Validation auc = 0.745070
Epoch 120
Validation auc = 0.746009
Epoch 121
Validation auc = 0.746479
Epoch 122
Validation auc = 0.751643
Epoch 123
Validation auc = 0.752582
Epoch 124
Loss = 2.8059e-02, PNorm = 108.6388, GNorm = 1.6008, lr_0 = 8.1870e-04
Validation auc = 0.750704
Epoch 125
Validation auc = 0.746009
Epoch 126
Validation auc = 0.741784
Epoch 127
Validation auc = 0.740845
Epoch 128
Validation auc = 0.740845
Epoch 129
Loss = 1.2381e-02, PNorm = 108.8110, GNorm = 0.8659, lr_0 = 8.1170e-04
Validation auc = 0.743662
Epoch 130
Validation auc = 0.748357
Epoch 131
Validation auc = 0.754930
Epoch 132
Validation auc = 0.756338
Epoch 133
Validation auc = 0.756808
Epoch 134
Loss = 6.2663e-03, PNorm = 108.9535, GNorm = 0.0134, lr_0 = 8.0476e-04
Validation auc = 0.756338
Epoch 135
Validation auc = 0.758216
Epoch 136
Validation auc = 0.758685
Epoch 137
Validation auc = 0.760094
Epoch 138
Validation auc = 0.750235
Epoch 139
Loss = 2.5946e-02, PNorm = 109.0937, GNorm = 2.3056, lr_0 = 7.9788e-04
Validation auc = 0.749765
Epoch 140
Validation auc = 0.739437
Epoch 141
Validation auc = 0.730047
Epoch 142
Validation auc = 0.725822
Epoch 143
Validation auc = 0.727230
Epoch 144
Loss = 1.1684e-03, PNorm = 109.2421, GNorm = 0.1534, lr_0 = 7.9106e-04
Validation auc = 0.730516
Epoch 145
Validation auc = 0.730986
Epoch 146
Validation auc = 0.723944
Epoch 147
Validation auc = 0.722535
Epoch 148
Validation auc = 0.727230
Epoch 149
Loss = 6.8939e-03, PNorm = 109.3563, GNorm = 0.2079, lr_0 = 7.8430e-04
Validation auc = 0.737089
Epoch 150
Validation auc = 0.743662
Epoch 151
Validation auc = 0.754460
Epoch 152
Validation auc = 0.758685
Epoch 153
Validation auc = 0.761972
Epoch 154
Loss = 3.8039e-03, PNorm = 109.4956, GNorm = 0.3071, lr_0 = 7.7759e-04
Validation auc = 0.766197
Epoch 155
Validation auc = 0.763380
Epoch 156
Validation auc = 0.757277
Epoch 157
Validation auc = 0.763380
Epoch 158
Validation auc = 0.761033
Epoch 159
Loss = 4.0385e-03, PNorm = 109.6406, GNorm = 0.2484, lr_0 = 7.7095e-04
Validation auc = 0.745070
Epoch 160
Validation auc = 0.751174
Epoch 161
Validation auc = 0.753521
Epoch 162
Validation auc = 0.749765
Epoch 163
Validation auc = 0.743662
Epoch 164
Loss = 1.9032e-02, PNorm = 109.8891, GNorm = 1.2250, lr_0 = 7.6436e-04
Validation auc = 0.743662
Epoch 165
Validation auc = 0.737559
Epoch 166
Validation auc = 0.742723
Epoch 167
Validation auc = 0.747887
Epoch 168
Validation auc = 0.751643
Epoch 169
Loss = 1.4816e-02, PNorm = 110.0711, GNorm = 2.0403, lr_0 = 7.5782e-04
Validation auc = 0.746009
Epoch 170
Validation auc = 0.730516
Epoch 171
Validation auc = 0.726761
Epoch 172
Validation auc = 0.722535
Epoch 173
Validation auc = 0.721596
Epoch 174
Loss = 3.1793e-02, PNorm = 110.2017, GNorm = 3.1068, lr_0 = 7.5134e-04
Validation auc = 0.727230
Epoch 175
Validation auc = 0.730516
Epoch 176
Validation auc = 0.737089
Epoch 177
Validation auc = 0.736150
Epoch 178
Validation auc = 0.730986
Epoch 179
Loss = 2.4191e-03, PNorm = 110.3399, GNorm = 0.6054, lr_0 = 7.4492e-04
Validation auc = 0.726761
Epoch 180
Validation auc = 0.727230
Epoch 181
Validation auc = 0.729577
Epoch 182
Validation auc = 0.730047
Epoch 183
Validation auc = 0.731925
Epoch 184
Loss = 6.3985e-03, PNorm = 110.4588, GNorm = 0.0475, lr_0 = 7.3855e-04
Validation auc = 0.731925
Epoch 185
Validation auc = 0.737559
Epoch 186
Validation auc = 0.732864
Epoch 187
Validation auc = 0.733333
Epoch 188
Validation auc = 0.733333
Epoch 189
Loss = 1.8690e-03, PNorm = 110.5327, GNorm = 0.0439, lr_0 = 7.3224e-04
Validation auc = 0.732394
Epoch 190
Validation auc = 0.732864
Epoch 191
Validation auc = 0.734742
Epoch 192
Validation auc = 0.737559
Epoch 193
Validation auc = 0.733333
Epoch 194
Loss = 1.0936e-03, PNorm = 110.5940, GNorm = 0.0393, lr_0 = 7.2598e-04
Validation auc = 0.730516
Epoch 195
Validation auc = 0.735211
Epoch 196
Validation auc = 0.736150
Epoch 197
Validation auc = 0.736620
Epoch 198
Validation auc = 0.736620
Epoch 199
Loss = 1.2189e-03, PNorm = 110.6442, GNorm = 0.2041, lr_0 = 7.1977e-04
Validation auc = 0.738967
Epoch 200
Validation auc = 0.740376
Epoch 201
Validation auc = 0.738967
Epoch 202
Validation auc = 0.738967
Epoch 203
Validation auc = 0.738498
Epoch 204
Loss = 4.9297e-04, PNorm = 110.6758, GNorm = 0.0944, lr_0 = 7.1362e-04
Validation auc = 0.737089
Epoch 205
Validation auc = 0.737089
Epoch 206
Validation auc = 0.743662
Epoch 207
Validation auc = 0.743662
Epoch 208
Validation auc = 0.743192
Epoch 209
Loss = 6.8841e-04, PNorm = 110.7175, GNorm = 0.0377, lr_0 = 7.0752e-04
Validation auc = 0.740376
Epoch 210
Validation auc = 0.737559
Epoch 211
Validation auc = 0.733333
Epoch 212
Validation auc = 0.729108
Epoch 213
Validation auc = 0.728638
Epoch 214
Loss = 3.2582e-04, PNorm = 110.7512, GNorm = 0.0715, lr_0 = 7.0147e-04
Validation auc = 0.727700
Epoch 215
Validation auc = 0.726761
Epoch 216
Validation auc = 0.726761
Epoch 217
Validation auc = 0.726291
Epoch 218
Validation auc = 0.724413
Epoch 219
Loss = 1.4775e-04, PNorm = 110.7686, GNorm = 0.0156, lr_0 = 6.9548e-04
Validation auc = 0.721127
Epoch 220
Validation auc = 0.720188
Epoch 221
Validation auc = 0.720657
Epoch 222
Validation auc = 0.721596
Epoch 223
Validation auc = 0.721127
Epoch 224
Loss = 1.4059e-04, PNorm = 110.7820, GNorm = 0.0082, lr_0 = 6.8953e-04
Validation auc = 0.721596
Epoch 225
Validation auc = 0.722066
Epoch 226
Validation auc = 0.725352
Epoch 227
Validation auc = 0.723474
Epoch 228
Validation auc = 0.726761
Epoch 229
Loss = 2.0970e-03, PNorm = 110.8004, GNorm = 0.6728, lr_0 = 6.8364e-04
Validation auc = 0.724413
Epoch 230
Validation auc = 0.722535
Epoch 231
Validation auc = 0.715023
Epoch 232
Validation auc = 0.718779
Epoch 233
Validation auc = 0.712207
Epoch 234
Loss = 6.9694e-03, PNorm = 110.8505, GNorm = 1.2615, lr_0 = 6.7779e-04
Validation auc = 0.709859
Epoch 235
Validation auc = 0.719249
Epoch 236
Validation auc = 0.723474
Epoch 237
Validation auc = 0.727230
Epoch 238
Validation auc = 0.729108
Epoch 239
Loss = 1.7677e-03, PNorm = 110.9688, GNorm = 0.1780, lr_0 = 6.7200e-04
Validation auc = 0.731455
Epoch 240
Validation auc = 0.730047
Epoch 241
Validation auc = 0.727230
Epoch 242
Validation auc = 0.725822
Epoch 243
Validation auc = 0.725822
Epoch 244
Loss = 2.2087e-03, PNorm = 111.0495, GNorm = 0.1242, lr_0 = 6.6625e-04
Validation auc = 0.723474
Epoch 245
Validation auc = 0.724883
Epoch 246
Validation auc = 0.724413
Epoch 247
Validation auc = 0.724883
Epoch 248
Validation auc = 0.725822
Epoch 249
Loss = 1.3438e-03, PNorm = 111.0989, GNorm = 0.1525, lr_0 = 6.6056e-04
Validation auc = 0.726761
Epoch 250
Validation auc = 0.727230
Epoch 251
Validation auc = 0.725822
Epoch 252
Validation auc = 0.725822
Epoch 253
Validation auc = 0.723944
Epoch 254
Loss = 4.0959e-04, PNorm = 111.1375, GNorm = 0.0139, lr_0 = 6.5491e-04
Validation auc = 0.721596
Epoch 255
Validation auc = 0.719249
Epoch 256
Validation auc = 0.719718
Epoch 257
Validation auc = 0.718779
Epoch 258
Validation auc = 0.719249
Epoch 259
Loss = 2.1452e-04, PNorm = 111.1636, GNorm = 0.0142, lr_0 = 6.4931e-04
Validation auc = 0.717371
Epoch 260
Validation auc = 0.717840
Epoch 261
Validation auc = 0.718310
Epoch 262
Validation auc = 0.718779
Epoch 263
Validation auc = 0.718779
Epoch 264
Loss = 3.9322e-04, PNorm = 111.1830, GNorm = 0.0178, lr_0 = 6.4376e-04
Validation auc = 0.716901
Epoch 265
Validation auc = 0.715962
Epoch 266
Validation auc = 0.714554
Epoch 267
Validation auc = 0.714554
Epoch 268
Validation auc = 0.714554
Epoch 269
Loss = 8.8747e-04, PNorm = 111.2014, GNorm = 0.0089, lr_0 = 6.3826e-04
Validation auc = 0.714554
Epoch 270
Validation auc = 0.714554
Epoch 271
Validation auc = 0.714554
Epoch 272
Validation auc = 0.714554
Epoch 273
Validation auc = 0.714085
Epoch 274
Loss = 1.7926e-03, PNorm = 111.2141, GNorm = 0.4886, lr_0 = 6.3280e-04
Validation auc = 0.713146
Epoch 275
Validation auc = 0.713146
Epoch 276
Validation auc = 0.711737
Epoch 277
Validation auc = 0.712207
Epoch 278
Validation auc = 0.710798
Epoch 279
Loss = 3.8466e-05, PNorm = 111.2193, GNorm = 0.0014, lr_0 = 6.2739e-04
Validation auc = 0.714085
Epoch 280
Validation auc = 0.716432
Epoch 281
Validation auc = 0.718779
Epoch 282
Validation auc = 0.715023
Epoch 283
Validation auc = 0.715962
Epoch 284
Loss = 6.2267e-02, PNorm = 111.2289, GNorm = 6.0889, lr_0 = 6.2203e-04
Validation auc = 0.700469
Epoch 285
Validation auc = 0.678404
Epoch 286
Validation auc = 0.691080
Epoch 287
Validation auc = 0.696714
Epoch 288
Validation auc = 0.697653
Epoch 289
Loss = 1.1080e-01, PNorm = 111.3855, GNorm = 8.1704, lr_0 = 6.1671e-04
Validation auc = 0.704225
Epoch 290
Validation auc = 0.715493
Epoch 291
Validation auc = 0.709859
Epoch 292
Validation auc = 0.713615
Epoch 293
Validation auc = 0.710798
Epoch 294
Loss = 4.5266e-02, PNorm = 111.6814, GNorm = 6.6262, lr_0 = 6.1144e-04
Validation auc = 0.710329
Epoch 295
Validation auc = 0.702817
Epoch 296
Validation auc = 0.686854
Epoch 297
Validation auc = 0.676056
Epoch 298
Validation auc = 0.670423
Epoch 299
Loss = 1.3673e-01, PNorm = 112.0803, GNorm = 2.7613, lr_0 = 6.0621e-04
Validation auc = 0.676995
Model 0 best validation auc = 0.766197 on epoch 154
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.881999
Ensemble test auc = 0.881999
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.651643
Epoch 1
Validation auc = 0.642723
Epoch 2
Validation auc = 0.691549
Epoch 3
Validation auc = 0.672300
Epoch 4
Loss = 6.3384e-01, PNorm = 100.5497, GNorm = 8.9660, lr_0 = 6.5000e-04
Validation auc = 0.672770
Epoch 5
Validation auc = 0.630516
Epoch 6
Validation auc = 0.682160
Epoch 7
Validation auc = 0.653991
Epoch 8
Validation auc = 0.655869
Epoch 9
Loss = 4.6396e-01, PNorm = 100.9029, GNorm = 1.2095, lr_0 = 9.9743e-04
Validation auc = 0.608920
Epoch 10
Validation auc = 0.621127
Epoch 11
Validation auc = 0.617371
Epoch 12
Validation auc = 0.660094
Epoch 13
Validation auc = 0.671831
Epoch 14
Loss = 3.7406e-01, PNorm = 101.2597, GNorm = 4.7614, lr_0 = 9.8890e-04
Validation auc = 0.635681
Epoch 15
Validation auc = 0.655869
Epoch 16
Validation auc = 0.621127
Epoch 17
Validation auc = 0.639437
Epoch 18
Validation auc = 0.665728
Epoch 19
Loss = 3.5660e-01, PNorm = 101.5531, GNorm = 2.5863, lr_0 = 9.8045e-04
Validation auc = 0.663850
Epoch 20
Validation auc = 0.683568
Epoch 21
Validation auc = 0.700469
Epoch 22
Validation auc = 0.707512
Epoch 23
Validation auc = 0.721596
Epoch 24
Loss = 2.0926e-01, PNorm = 101.8463, GNorm = 2.5091, lr_0 = 9.7207e-04
Validation auc = 0.699061
Epoch 25
Validation auc = 0.692958
Epoch 26
Validation auc = 0.699531
Epoch 27
Validation auc = 0.715023
Epoch 28
Validation auc = 0.720188
Epoch 29
Loss = 1.8708e-01, PNorm = 102.1581, GNorm = 4.6268, lr_0 = 9.6376e-04
Validation auc = 0.718310
Epoch 30
Validation auc = 0.715493
Epoch 31
Validation auc = 0.709390
Epoch 32
Validation auc = 0.708920
Epoch 33
Validation auc = 0.698122
Epoch 34
Loss = 2.0524e-01, PNorm = 102.4916, GNorm = 5.0077, lr_0 = 9.5552e-04
Validation auc = 0.706573
Epoch 35
Validation auc = 0.711268
Epoch 36
Validation auc = 0.702347
Epoch 37
Validation auc = 0.694366
Epoch 38
Validation auc = 0.681690
Epoch 39
Loss = 1.8672e-01, PNorm = 102.8343, GNorm = 2.4993, lr_0 = 9.4735e-04
Validation auc = 0.699061
Epoch 40
Validation auc = 0.692488
Epoch 41
Validation auc = 0.690141
Epoch 42
Validation auc = 0.692488
Epoch 43
Validation auc = 0.690610
Epoch 44
Loss = 2.8599e-01, PNorm = 103.1933, GNorm = 2.1648, lr_0 = 9.3925e-04
Validation auc = 0.670423
Epoch 45
Validation auc = 0.689202
Epoch 46
Validation auc = 0.692019
Epoch 47
Validation auc = 0.695305
Epoch 48
Validation auc = 0.667606
Epoch 49
Loss = 1.9571e-01, PNorm = 103.6200, GNorm = 5.2904, lr_0 = 9.3122e-04
Validation auc = 0.682160
Epoch 50
Validation auc = 0.676056
Epoch 51
Validation auc = 0.679343
Epoch 52
Validation auc = 0.692019
Epoch 53
Validation auc = 0.686854
Epoch 54
Loss = 1.4597e-01, PNorm = 104.0336, GNorm = 0.9094, lr_0 = 9.2326e-04
Validation auc = 0.676526
Epoch 55
Validation auc = 0.652582
Epoch 56
Validation auc = 0.647887
Epoch 57
Validation auc = 0.657277
Epoch 58
Validation auc = 0.699061
Epoch 59
Loss = 1.9342e-01, PNorm = 104.4021, GNorm = 7.3283, lr_0 = 9.1537e-04
Validation auc = 0.692488
Epoch 60
Validation auc = 0.703286
Epoch 61
Validation auc = 0.691080
Epoch 62
Validation auc = 0.678873
Epoch 63
Validation auc = 0.670423
Epoch 64
Loss = 8.3958e-02, PNorm = 104.8091, GNorm = 2.8938, lr_0 = 9.0754e-04
Validation auc = 0.659155
Epoch 65
Validation auc = 0.668545
Epoch 66
Validation auc = 0.670892
Epoch 67
Validation auc = 0.664319
Epoch 68
Validation auc = 0.651174
Epoch 69
Loss = 1.0303e-01, PNorm = 105.1585, GNorm = 6.0388, lr_0 = 8.9978e-04
Validation auc = 0.652113
Epoch 70
Validation auc = 0.669953
Epoch 71
Validation auc = 0.674648
Epoch 72
Validation auc = 0.667606
Epoch 73
Validation auc = 0.654930
Epoch 74
Loss = 1.5668e-01, PNorm = 105.4641, GNorm = 8.4677, lr_0 = 8.9209e-04
Validation auc = 0.658685
Epoch 75
Validation auc = 0.670423
Epoch 76
Validation auc = 0.665728
Epoch 77
Validation auc = 0.700000
Epoch 78
Validation auc = 0.708920
Epoch 79
Loss = 1.7315e-01, PNorm = 105.8084, GNorm = 9.5675, lr_0 = 8.8447e-04
Validation auc = 0.708451
Epoch 80
Validation auc = 0.694366
Epoch 81
Validation auc = 0.664319
Epoch 82
Validation auc = 0.642254
Epoch 83
Validation auc = 0.633333
Epoch 84
Loss = 9.1029e-02, PNorm = 106.1792, GNorm = 1.8582, lr_0 = 8.7691e-04
Validation auc = 0.678873
Epoch 85
Validation auc = 0.682629
Epoch 86
Validation auc = 0.682160
Epoch 87
Validation auc = 0.691080
Epoch 88
Validation auc = 0.688732
Epoch 89
Loss = 2.6976e-02, PNorm = 106.5415, GNorm = 0.7107, lr_0 = 8.6941e-04
Validation auc = 0.688732
Epoch 90
Validation auc = 0.688732
Epoch 91
Validation auc = 0.691080
Epoch 92
Validation auc = 0.683568
Epoch 93
Validation auc = 0.679812
Epoch 94
Loss = 3.0764e-02, PNorm = 106.8274, GNorm = 2.1847, lr_0 = 8.6198e-04
Validation auc = 0.675117
Epoch 95
Validation auc = 0.679812
Epoch 96
Validation auc = 0.685446
Epoch 97
Validation auc = 0.711737
Epoch 98
Validation auc = 0.721596
Epoch 99
Loss = 1.4882e-01, PNorm = 107.0741, GNorm = 8.2009, lr_0 = 8.5461e-04
Validation auc = 0.728638
Epoch 100
Validation auc = 0.719718
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.689671
Epoch 103
Validation auc = 0.677934
Epoch 104
Loss = 1.1022e-01, PNorm = 107.4762, GNorm = 7.2773, lr_0 = 8.4730e-04
Validation auc = 0.677465
Epoch 105
Validation auc = 0.692019
Epoch 106
Validation auc = 0.684507
Epoch 107
Validation auc = 0.698592
Epoch 108
Validation auc = 0.700939
Epoch 109
Loss = 2.7797e-02, PNorm = 107.8794, GNorm = 1.7155, lr_0 = 8.4006e-04
Validation auc = 0.693897
Epoch 110
Validation auc = 0.692019
Epoch 111
Validation auc = 0.699061
Epoch 112
Validation auc = 0.711737
Epoch 113
Validation auc = 0.720657
Epoch 114
Loss = 6.4885e-02, PNorm = 108.1838, GNorm = 1.4458, lr_0 = 8.3288e-04
Validation auc = 0.716432
Epoch 115
Validation auc = 0.716432
Epoch 116
Validation auc = 0.707042
Epoch 117
Validation auc = 0.687793
Epoch 118
Validation auc = 0.664319
Epoch 119
Loss = 3.3083e-02, PNorm = 108.4334, GNorm = 0.2453, lr_0 = 8.2576e-04
Validation auc = 0.622535
Epoch 120
Validation auc = 0.619718
Epoch 121
Validation auc = 0.675587
Epoch 122
Validation auc = 0.685446
Epoch 123
Validation auc = 0.684038
Epoch 124
Loss = 1.5091e-02, PNorm = 108.7459, GNorm = 1.1078, lr_0 = 8.1870e-04
Validation auc = 0.670423
Epoch 125
Validation auc = 0.659155
Epoch 126
Validation auc = 0.657277
Epoch 127
Validation auc = 0.659155
Epoch 128
Validation auc = 0.677465
Epoch 129
Loss = 9.7294e-03, PNorm = 109.0446, GNorm = 0.6472, lr_0 = 8.1170e-04
Validation auc = 0.698122
Epoch 130
Validation auc = 0.710798
Epoch 131
Validation auc = 0.718779
Epoch 132
Validation auc = 0.721127
Epoch 133
Validation auc = 0.720657
Epoch 134
Loss = 2.3579e-02, PNorm = 109.2416, GNorm = 0.5594, lr_0 = 8.0476e-04
Validation auc = 0.722066
Epoch 135
Validation auc = 0.723005
Epoch 136
Validation auc = 0.719249
Epoch 137
Validation auc = 0.720657
Epoch 138
Validation auc = 0.711737
Epoch 139
Loss = 5.8224e-02, PNorm = 109.3915, GNorm = 0.9798, lr_0 = 7.9788e-04
Validation auc = 0.713146
Epoch 140
Validation auc = 0.706573
Epoch 141
Validation auc = 0.694836
Epoch 142
Validation auc = 0.697183
Epoch 143
Validation auc = 0.696714
Epoch 144
Loss = 6.4589e-03, PNorm = 109.5092, GNorm = 1.2169, lr_0 = 7.9106e-04
Validation auc = 0.702347
Epoch 145
Validation auc = 0.711737
Epoch 146
Validation auc = 0.710329
Epoch 147
Validation auc = 0.704695
Epoch 148
Validation auc = 0.697653
Epoch 149
Loss = 9.0445e-03, PNorm = 109.6170, GNorm = 0.1738, lr_0 = 7.8430e-04
Validation auc = 0.698592
Epoch 150
Validation auc = 0.700000
Epoch 151
Validation auc = 0.700939
Epoch 152
Validation auc = 0.701878
Epoch 153
Validation auc = 0.700939
Epoch 154
Loss = 1.8166e-03, PNorm = 109.6919, GNorm = 0.1925, lr_0 = 7.7759e-04
Validation auc = 0.703286
Epoch 155
Validation auc = 0.705634
Epoch 156
Validation auc = 0.706573
Epoch 157
Validation auc = 0.713615
Epoch 158
Validation auc = 0.719718
Epoch 159
Loss = 4.2296e-02, PNorm = 109.7597, GNorm = 2.2567, lr_0 = 7.7095e-04
Validation auc = 0.732864
Epoch 160
Validation auc = 0.738498
Epoch 161
Validation auc = 0.734742
Epoch 162
Validation auc = 0.730047
Epoch 163
Validation auc = 0.724883
Epoch 164
Loss = 1.2609e-03, PNorm = 109.8778, GNorm = 0.0819, lr_0 = 7.6436e-04
Validation auc = 0.722535
Epoch 165
Validation auc = 0.716432
Epoch 166
Validation auc = 0.714085
Epoch 167
Validation auc = 0.709859
Epoch 168
Validation auc = 0.706103
Epoch 169
Loss = 1.4789e-03, PNorm = 109.9631, GNorm = 0.0901, lr_0 = 7.5782e-04
Validation auc = 0.705634
Epoch 170
Validation auc = 0.702817
Epoch 171
Validation auc = 0.700939
Epoch 172
Validation auc = 0.701408
Epoch 173
Validation auc = 0.699531
Epoch 174
Loss = 2.6985e-03, PNorm = 110.0412, GNorm = 0.0399, lr_0 = 7.5134e-04
Validation auc = 0.699061
Epoch 175
Validation auc = 0.701408
Epoch 176
Validation auc = 0.713146
Epoch 177
Validation auc = 0.720657
Epoch 178
Validation auc = 0.726291
Epoch 179
Loss = 2.4577e-02, PNorm = 110.0902, GNorm = 1.8562, lr_0 = 7.4492e-04
Validation auc = 0.728638
Epoch 180
Validation auc = 0.733803
Epoch 181
Validation auc = 0.732864
Epoch 182
Validation auc = 0.729577
Epoch 183
Validation auc = 0.723474
Epoch 184
Loss = 6.3611e-03, PNorm = 110.1461, GNorm = 0.8336, lr_0 = 7.3855e-04
Validation auc = 0.728169
Epoch 185
Validation auc = 0.731925
Epoch 186
Validation auc = 0.735211
Epoch 187
Validation auc = 0.724413
Epoch 188
Validation auc = 0.712676
Epoch 189
Loss = 1.7380e-03, PNorm = 110.2434, GNorm = 0.3611, lr_0 = 7.3224e-04
Validation auc = 0.708920
Epoch 190
Validation auc = 0.710329
Epoch 191
Validation auc = 0.723005
Epoch 192
Validation auc = 0.731455
Epoch 193
Validation auc = 0.740845
Epoch 194
Loss = 1.3427e-02, PNorm = 110.3956, GNorm = 2.1246, lr_0 = 7.2598e-04
Validation auc = 0.739437
Epoch 195
Validation auc = 0.737089
Epoch 196
Validation auc = 0.738498
Epoch 197
Validation auc = 0.734742
Epoch 198
Validation auc = 0.739437
Epoch 199
Loss = 7.6039e-02, PNorm = 110.5454, GNorm = 2.1259, lr_0 = 7.1977e-04
Validation auc = 0.722066
Epoch 200
Validation auc = 0.720188
Epoch 201
Validation auc = 0.723005
Epoch 202
Validation auc = 0.732864
Epoch 203
Validation auc = 0.736150
Epoch 204
Loss = 4.0458e-02, PNorm = 110.7186, GNorm = 1.0717, lr_0 = 7.1362e-04
Validation auc = 0.726761
Epoch 205
Validation auc = 0.732864
Epoch 206
Validation auc = 0.736620
Epoch 207
Validation auc = 0.733333
Epoch 208
Validation auc = 0.746009
Epoch 209
Loss = 3.6884e-02, PNorm = 110.8971, GNorm = 0.1805, lr_0 = 7.0752e-04
Validation auc = 0.755399
Epoch 210
Validation auc = 0.758685
Epoch 211
Validation auc = 0.748357
Epoch 212
Validation auc = 0.739437
Epoch 213
Validation auc = 0.737559
Epoch 214
Loss = 6.4267e-03, PNorm = 111.1232, GNorm = 0.3604, lr_0 = 7.0147e-04
Validation auc = 0.742254
Epoch 215
Validation auc = 0.739437
Epoch 216
Validation auc = 0.730986
Epoch 217
Validation auc = 0.731455
Epoch 218
Validation auc = 0.736620
Epoch 219
Loss = 1.4081e-02, PNorm = 111.2759, GNorm = 1.0947, lr_0 = 6.9548e-04
Validation auc = 0.733333
Epoch 220
Validation auc = 0.728169
Epoch 221
Validation auc = 0.722066
Epoch 222
Validation auc = 0.712676
Epoch 223
Validation auc = 0.700939
Epoch 224
Loss = 4.5713e-02, PNorm = 111.3879, GNorm = 3.9802, lr_0 = 6.8953e-04
Validation auc = 0.696244
Epoch 225
Validation auc = 0.691080
Epoch 226
Validation auc = 0.692488
Epoch 227
Validation auc = 0.695305
Epoch 228
Validation auc = 0.700469
Epoch 229
Loss = 1.2998e-02, PNorm = 111.5003, GNorm = 0.2300, lr_0 = 6.8364e-04
Validation auc = 0.706573
Epoch 230
Validation auc = 0.707981
Epoch 231
Validation auc = 0.704695
Epoch 232
Validation auc = 0.697653
Epoch 233
Validation auc = 0.690610
Epoch 234
Loss = 1.1369e-02, PNorm = 111.5940, GNorm = 1.0804, lr_0 = 6.7779e-04
Validation auc = 0.688732
Epoch 235
Validation auc = 0.693427
Epoch 236
Validation auc = 0.696244
Epoch 237
Validation auc = 0.700000
Epoch 238
Validation auc = 0.701878
Epoch 239
Loss = 1.7978e-02, PNorm = 111.7049, GNorm = 2.1478, lr_0 = 6.7200e-04
Validation auc = 0.699531
Epoch 240
Validation auc = 0.707042
Epoch 241
Validation auc = 0.714085
Epoch 242
Validation auc = 0.718779
Epoch 243
Validation auc = 0.717371
Epoch 244
Loss = 1.9994e-03, PNorm = 111.8027, GNorm = 0.2370, lr_0 = 6.6625e-04
Validation auc = 0.717371
Epoch 245
Validation auc = 0.716432
Epoch 246
Validation auc = 0.717371
Epoch 247
Validation auc = 0.718779
Epoch 248
Validation auc = 0.720188
Epoch 249
Loss = 2.2664e-03, PNorm = 111.8850, GNorm = 0.2992, lr_0 = 6.6056e-04
Validation auc = 0.720657
Epoch 250
Validation auc = 0.721127
Epoch 251
Validation auc = 0.715962
Epoch 252
Validation auc = 0.714554
Epoch 253
Validation auc = 0.709390
Epoch 254
Loss = 1.7275e-04, PNorm = 111.9418, GNorm = 0.0093, lr_0 = 6.5491e-04
Validation auc = 0.703756
Epoch 255
Validation auc = 0.700939
Epoch 256
Validation auc = 0.695305
Epoch 257
Validation auc = 0.693897
Epoch 258
Validation auc = 0.692019
Epoch 259
Loss = 3.6741e-02, PNorm = 111.9716, GNorm = 2.8419, lr_0 = 6.4931e-04
Validation auc = 0.702817
Epoch 260
Validation auc = 0.713615
Epoch 261
Validation auc = 0.721127
Epoch 262
Validation auc = 0.723944
Epoch 263
Validation auc = 0.724413
Epoch 264
Loss = 3.2160e-03, PNorm = 112.0111, GNorm = 0.4122, lr_0 = 6.4376e-04
Validation auc = 0.726291
Epoch 265
Validation auc = 0.725352
Epoch 266
Validation auc = 0.725352
Epoch 267
Validation auc = 0.723474
Epoch 268
Validation auc = 0.723944
Epoch 269
Loss = 8.7369e-02, PNorm = 112.0463, GNorm = 0.0132, lr_0 = 6.3826e-04
Validation auc = 0.728169
Epoch 270
Validation auc = 0.728169
Epoch 271
Validation auc = 0.724413
Epoch 272
Validation auc = 0.716432
Epoch 273
Validation auc = 0.694836
Epoch 274
Loss = 3.6122e-02, PNorm = 112.1386, GNorm = 2.0802, lr_0 = 6.3280e-04
Validation auc = 0.686854
Epoch 275
Validation auc = 0.695775
Epoch 276
Validation auc = 0.697653
Epoch 277
Validation auc = 0.698592
Epoch 278
Validation auc = 0.700000
Epoch 279
Loss = 2.5503e-03, PNorm = 112.2577, GNorm = 0.0643, lr_0 = 6.2739e-04
Validation auc = 0.700939
Epoch 280
Validation auc = 0.703756
Epoch 281
Validation auc = 0.705164
Epoch 282
Validation auc = 0.723474
Epoch 283
Validation auc = 0.735211
Epoch 284
Loss = 2.8031e-02, PNorm = 112.3529, GNorm = 0.1354, lr_0 = 6.2203e-04
Validation auc = 0.754930
Epoch 285
Validation auc = 0.759155
Epoch 286
Validation auc = 0.765728
Epoch 287
Validation auc = 0.759155
Epoch 288
Validation auc = 0.755869
Epoch 289
Loss = 4.7138e-03, PNorm = 112.4639, GNorm = 0.2251, lr_0 = 6.1671e-04
Validation auc = 0.750704
Epoch 290
Validation auc = 0.751643
Epoch 291
Validation auc = 0.751174
Epoch 292
Validation auc = 0.757277
Epoch 293
Validation auc = 0.756808
Epoch 294
Loss = 2.0885e-04, PNorm = 112.5447, GNorm = 0.0141, lr_0 = 6.1144e-04
Validation auc = 0.753991
Epoch 295
Validation auc = 0.754930
Epoch 296
Validation auc = 0.752582
Epoch 297
Validation auc = 0.752582
Epoch 298
Validation auc = 0.753052
Epoch 299
Loss = 2.8955e-04, PNorm = 112.5828, GNorm = 0.0292, lr_0 = 6.0621e-04
Validation auc = 0.753991
Model 0 best validation auc = 0.765728 on epoch 286
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.888621
Ensemble test auc = 0.888621
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.686854
Epoch 1
Validation auc = 0.646948
Epoch 2
Validation auc = 0.679812
Epoch 3
Validation auc = 0.656338
Epoch 4
Loss = 6.1341e-01, PNorm = 100.5463, GNorm = 9.1628, lr_0 = 6.5000e-04
Validation auc = 0.623474
Epoch 5
Validation auc = 0.629108
Epoch 6
Validation auc = 0.612207
Epoch 7
Validation auc = 0.674648
Epoch 8
Validation auc = 0.618779
Epoch 9
Loss = 5.4815e-01, PNorm = 100.9004, GNorm = 4.0674, lr_0 = 9.9743e-04
Validation auc = 0.670423
Epoch 10
Validation auc = 0.604225
Epoch 11
Validation auc = 0.686854
Epoch 12
Validation auc = 0.685446
Epoch 13
Validation auc = 0.692488
Epoch 14
Loss = 5.3964e-01, PNorm = 101.2835, GNorm = 6.8432, lr_0 = 9.8890e-04
Validation auc = 0.698122
Epoch 15
Validation auc = 0.697183
Epoch 16
Validation auc = 0.697183
Epoch 17
Validation auc = 0.699061
Epoch 18
Validation auc = 0.696244
Epoch 19
Loss = 3.0547e-01, PNorm = 101.5972, GNorm = 3.9842, lr_0 = 9.8045e-04
Validation auc = 0.692958
Epoch 20
Validation auc = 0.703756
Epoch 21
Validation auc = 0.698592
Epoch 22
Validation auc = 0.670892
Epoch 23
Validation auc = 0.661972
Epoch 24
Loss = 2.7792e-01, PNorm = 101.9179, GNorm = 2.0744, lr_0 = 9.7207e-04
Validation auc = 0.679343
Epoch 25
Validation auc = 0.692019
Epoch 26
Validation auc = 0.695775
Epoch 27
Validation auc = 0.701408
Epoch 28
Validation auc = 0.730986
Epoch 29
Loss = 2.7569e-01, PNorm = 102.2260, GNorm = 10.2105, lr_0 = 9.6376e-04
Validation auc = 0.733803
Epoch 30
Validation auc = 0.751174
Epoch 31
Validation auc = 0.732394
Epoch 32
Validation auc = 0.713146
Epoch 33
Validation auc = 0.714554
Epoch 34
Loss = 2.9169e-01, PNorm = 102.5533, GNorm = 4.0514, lr_0 = 9.5552e-04
Validation auc = 0.710798
Epoch 35
Validation auc = 0.726761
Epoch 36
Validation auc = 0.734742
Epoch 37
Validation auc = 0.717371
Epoch 38
Validation auc = 0.713146
Epoch 39
Loss = 1.6729e-01, PNorm = 102.9248, GNorm = 0.9374, lr_0 = 9.4735e-04
Validation auc = 0.677934
Epoch 40
Validation auc = 0.688263
Epoch 41
Validation auc = 0.703286
Epoch 42
Validation auc = 0.708920
Epoch 43
Validation auc = 0.705634
Epoch 44
Loss = 2.3903e-01, PNorm = 103.2687, GNorm = 5.2738, lr_0 = 9.3925e-04
Validation auc = 0.700000
Epoch 45
Validation auc = 0.707042
Epoch 46
Validation auc = 0.698592
Epoch 47
Validation auc = 0.684038
Epoch 48
Validation auc = 0.669014
Epoch 49
Loss = 1.4934e-01, PNorm = 103.6492, GNorm = 1.0478, lr_0 = 9.3122e-04
Validation auc = 0.654930
Epoch 50
Validation auc = 0.661033
Epoch 51
Validation auc = 0.684977
Epoch 52
Validation auc = 0.701878
Epoch 53
Validation auc = 0.713146
Epoch 54
Loss = 1.3186e-01, PNorm = 104.0101, GNorm = 2.7648, lr_0 = 9.2326e-04
Validation auc = 0.702347
Epoch 55
Validation auc = 0.693427
Epoch 56
Validation auc = 0.675117
Epoch 57
Validation auc = 0.644131
Epoch 58
Validation auc = 0.614085
Epoch 59
Loss = 1.0958e-01, PNorm = 104.3471, GNorm = 3.1536, lr_0 = 9.1537e-04
Validation auc = 0.607042
Epoch 60
Validation auc = 0.616432
Epoch 61
Validation auc = 0.647418
Epoch 62
Validation auc = 0.664319
Epoch 63
Validation auc = 0.675117
Epoch 64
Loss = 5.6125e-02, PNorm = 104.6355, GNorm = 1.0978, lr_0 = 9.0754e-04
Validation auc = 0.675587
Epoch 65
Validation auc = 0.680282
Epoch 66
Validation auc = 0.691549
Epoch 67
Validation auc = 0.696714
Epoch 68
Validation auc = 0.704695
Epoch 69
Loss = 3.3853e-02, PNorm = 104.9173, GNorm = 0.8513, lr_0 = 8.9978e-04
Validation auc = 0.684977
Epoch 70
Validation auc = 0.698122
Epoch 71
Validation auc = 0.665728
Epoch 72
Validation auc = 0.639906
Epoch 73
Validation auc = 0.620657
Epoch 74
Loss = 5.3174e-02, PNorm = 105.2793, GNorm = 1.1420, lr_0 = 8.9209e-04
Validation auc = 0.601878
Epoch 75
Validation auc = 0.600939
Epoch 76
Validation auc = 0.606103
Epoch 77
Validation auc = 0.653521
Epoch 78
Validation auc = 0.647418
Epoch 79
Loss = 1.2028e-01, PNorm = 105.6775, GNorm = 5.6819, lr_0 = 8.8447e-04
Validation auc = 0.654460
Epoch 80
Validation auc = 0.660563
Epoch 81
Validation auc = 0.675587
Epoch 82
Validation auc = 0.701408
Epoch 83
Validation auc = 0.723474
Epoch 84
Loss = 6.5074e-02, PNorm = 106.0472, GNorm = 2.8462, lr_0 = 8.7691e-04
Validation auc = 0.729577
Epoch 85
Validation auc = 0.724883
Epoch 86
Validation auc = 0.701408
Epoch 87
Validation auc = 0.690141
Epoch 88
Validation auc = 0.696714
Epoch 89
Loss = 1.4673e-01, PNorm = 106.4078, GNorm = 4.8171, lr_0 = 8.6941e-04
Validation auc = 0.721596
Epoch 90
Validation auc = 0.733333
Epoch 91
Validation auc = 0.728638
Epoch 92
Validation auc = 0.717371
Epoch 93
Validation auc = 0.715493
Epoch 94
Loss = 2.3634e-01, PNorm = 106.7947, GNorm = 1.4991, lr_0 = 8.6198e-04
Validation auc = 0.692488
Epoch 95
Validation auc = 0.694836
Epoch 96
Validation auc = 0.710329
Epoch 97
Validation auc = 0.742254
Epoch 98
Validation auc = 0.751174
Epoch 99
Loss = 7.4377e-02, PNorm = 107.1440, GNorm = 1.0978, lr_0 = 8.5461e-04
Validation auc = 0.757277
Epoch 100
Validation auc = 0.755869
Epoch 101
Validation auc = 0.744601
Epoch 102
Validation auc = 0.728169
Epoch 103
Validation auc = 0.726761
Epoch 104
Loss = 1.5094e-02, PNorm = 107.4814, GNorm = 0.4216, lr_0 = 8.4730e-04
Validation auc = 0.720657
Epoch 105
Validation auc = 0.717840
Epoch 106
Validation auc = 0.724883
Epoch 107
Validation auc = 0.725352
Epoch 108
Validation auc = 0.728638
Epoch 109
Loss = 2.2158e-02, PNorm = 107.7312, GNorm = 0.8920, lr_0 = 8.4006e-04
Validation auc = 0.726761
Epoch 110
Validation auc = 0.725352
Epoch 111
Validation auc = 0.721127
Epoch 112
Validation auc = 0.723944
Epoch 113
Validation auc = 0.729108
Epoch 114
Loss = 6.9738e-02, PNorm = 107.9947, GNorm = 2.4777, lr_0 = 8.3288e-04
Validation auc = 0.729108
Epoch 115
Validation auc = 0.720188
Epoch 116
Validation auc = 0.708920
Epoch 117
Validation auc = 0.704695
Epoch 118
Validation auc = 0.715962
Epoch 119
Loss = 1.0707e-01, PNorm = 108.2553, GNorm = 5.3106, lr_0 = 8.2576e-04
Validation auc = 0.723005
Epoch 120
Validation auc = 0.720657
Epoch 121
Validation auc = 0.727700
Epoch 122
Validation auc = 0.711268
Epoch 123
Validation auc = 0.712676
Epoch 124
Loss = 4.1582e-02, PNorm = 108.5246, GNorm = 2.5768, lr_0 = 8.1870e-04
Validation auc = 0.716901
Epoch 125
Validation auc = 0.723944
Epoch 126
Validation auc = 0.729577
Epoch 127
Validation auc = 0.736150
Epoch 128
Validation auc = 0.732394
Epoch 129
Loss = 4.3463e-02, PNorm = 108.7670, GNorm = 2.1262, lr_0 = 8.1170e-04
Validation auc = 0.728638
Epoch 130
Validation auc = 0.733333
Epoch 131
Validation auc = 0.727230
Epoch 132
Validation auc = 0.723944
Epoch 133
Validation auc = 0.705164
Epoch 134
Loss = 2.7502e-03, PNorm = 109.0241, GNorm = 0.0885, lr_0 = 8.0476e-04
Validation auc = 0.697653
Epoch 135
Validation auc = 0.699531
Epoch 136
Validation auc = 0.721127
Epoch 137
Validation auc = 0.726291
Epoch 138
Validation auc = 0.733333
Epoch 139
Loss = 1.4387e-02, PNorm = 109.2211, GNorm = 1.6547, lr_0 = 7.9788e-04
Validation auc = 0.733333
Epoch 140
Validation auc = 0.750235
Epoch 141
Validation auc = 0.734742
Epoch 142
Validation auc = 0.719718
Epoch 143
Validation auc = 0.722535
Epoch 144
Loss = 2.9293e-02, PNorm = 109.3596, GNorm = 2.4955, lr_0 = 7.9106e-04
Validation auc = 0.718310
Epoch 145
Validation auc = 0.700000
Epoch 146
Validation auc = 0.715493
Epoch 147
Validation auc = 0.723005
Epoch 148
Validation auc = 0.730986
Epoch 149
Loss = 1.2386e-02, PNorm = 109.4877, GNorm = 1.7049, lr_0 = 7.8430e-04
Validation auc = 0.737089
Epoch 150
Validation auc = 0.737089
Epoch 151
Validation auc = 0.738967
Epoch 152
Validation auc = 0.732394
Epoch 153
Validation auc = 0.734272
Epoch 154
Loss = 2.1836e-02, PNorm = 109.6450, GNorm = 0.2716, lr_0 = 7.7759e-04
Validation auc = 0.736150
Epoch 155
Validation auc = 0.738498
Epoch 156
Validation auc = 0.737089
Epoch 157
Validation auc = 0.735681
Epoch 158
Validation auc = 0.733803
Epoch 159
Loss = 7.0126e-03, PNorm = 109.7617, GNorm = 0.7798, lr_0 = 7.7095e-04
Validation auc = 0.730986
Epoch 160
Validation auc = 0.730986
Epoch 161
Validation auc = 0.727700
Epoch 162
Validation auc = 0.724413
Epoch 163
Validation auc = 0.722066
Epoch 164
Loss = 1.6273e-03, PNorm = 109.8481, GNorm = 0.1057, lr_0 = 7.6436e-04
Validation auc = 0.718779
Epoch 165
Validation auc = 0.713615
Epoch 166
Validation auc = 0.708451
Epoch 167
Validation auc = 0.704225
Epoch 168
Validation auc = 0.704695
Epoch 169
Loss = 1.9689e-03, PNorm = 109.9159, GNorm = 0.0669, lr_0 = 7.5782e-04
Validation auc = 0.702347
Epoch 170
Validation auc = 0.700939
Epoch 171
Validation auc = 0.700000
Epoch 172
Validation auc = 0.705164
Epoch 173
Validation auc = 0.707042
Epoch 174
Loss = 3.5491e-04, PNorm = 109.9694, GNorm = 0.0247, lr_0 = 7.5134e-04
Validation auc = 0.712207
Epoch 175
Validation auc = 0.713615
Epoch 176
Validation auc = 0.713146
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.713146
Epoch 179
Loss = 8.0342e-04, PNorm = 110.0027, GNorm = 0.0147, lr_0 = 7.4492e-04
Validation auc = 0.715493
Epoch 180
Validation auc = 0.715962
Epoch 181
Validation auc = 0.718310
Epoch 182
Validation auc = 0.719718
Epoch 183
Validation auc = 0.724413
Epoch 184
Loss = 1.8723e-04, PNorm = 110.0253, GNorm = 0.0149, lr_0 = 7.3855e-04
Validation auc = 0.724413
Epoch 185
Validation auc = 0.726761
Epoch 186
Validation auc = 0.730516
Epoch 187
Validation auc = 0.730516
Epoch 188
Validation auc = 0.728638
Epoch 189
Loss = 3.2292e-04, PNorm = 110.0477, GNorm = 0.0401, lr_0 = 7.3224e-04
Validation auc = 0.727700
Epoch 190
Validation auc = 0.725352
Epoch 191
Validation auc = 0.721596
Epoch 192
Validation auc = 0.718779
Epoch 193
Validation auc = 0.717371
Epoch 194
Loss = 7.8559e-04, PNorm = 110.0668, GNorm = 0.2513, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.720657
Epoch 196
Validation auc = 0.720188
Epoch 197
Validation auc = 0.721127
Epoch 198
Validation auc = 0.721127
Epoch 199
Loss = 8.9720e-05, PNorm = 110.0747, GNorm = 0.0098, lr_0 = 7.1977e-04
Validation auc = 0.721127
Epoch 200
Validation auc = 0.721127
Epoch 201
Validation auc = 0.721596
Epoch 202
Validation auc = 0.721596
Epoch 203
Validation auc = 0.722066
Epoch 204
Loss = 2.2140e-05, PNorm = 110.0820, GNorm = 0.0023, lr_0 = 7.1362e-04
Validation auc = 0.722066
Epoch 205
Validation auc = 0.721127
Epoch 206
Validation auc = 0.724413
Epoch 207
Validation auc = 0.723005
Epoch 208
Validation auc = 0.724413
Epoch 209
Loss = 1.7063e-04, PNorm = 110.0890, GNorm = 0.0150, lr_0 = 7.0752e-04
Validation auc = 0.724883
Epoch 210
Validation auc = 0.723944
Epoch 211
Validation auc = 0.723474
Epoch 212
Validation auc = 0.724413
Epoch 213
Validation auc = 0.723474
Epoch 214
Loss = 6.4455e-05, PNorm = 110.0973, GNorm = 0.0047, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.726291
Epoch 216
Validation auc = 0.727230
Epoch 217
Validation auc = 0.727700
Epoch 218
Validation auc = 0.729108
Epoch 219
Loss = 2.3764e-04, PNorm = 110.1022, GNorm = 0.0032, lr_0 = 6.9548e-04
Validation auc = 0.728638
Epoch 220
Validation auc = 0.728638
Epoch 221
Validation auc = 0.728638
Epoch 222
Validation auc = 0.727230
Epoch 223
Validation auc = 0.722535
Epoch 224
Loss = 1.1017e-04, PNorm = 110.1137, GNorm = 0.0168, lr_0 = 6.8953e-04
Validation auc = 0.721596
Epoch 225
Validation auc = 0.725352
Epoch 226
Validation auc = 0.725822
Epoch 227
Validation auc = 0.728638
Epoch 228
Validation auc = 0.717840
Epoch 229
Loss = 2.6211e-04, PNorm = 110.1635, GNorm = 0.0654, lr_0 = 6.8364e-04
Validation auc = 0.719718
Epoch 230
Validation auc = 0.721596
Epoch 231
Validation auc = 0.719718
Epoch 232
Validation auc = 0.717371
Epoch 233
Validation auc = 0.707981
Epoch 234
Loss = 2.5962e-04, PNorm = 110.2732, GNorm = 0.0537, lr_0 = 6.7779e-04
Validation auc = 0.707042
Epoch 235
Validation auc = 0.707981
Epoch 236
Validation auc = 0.715023
Epoch 237
Validation auc = 0.718310
Epoch 238
Validation auc = 0.720188
Epoch 239
Loss = 1.7589e-02, PNorm = 110.3571, GNorm = 2.7031, lr_0 = 6.7200e-04
Validation auc = 0.720188
Epoch 240
Validation auc = 0.718310
Epoch 241
Validation auc = 0.724413
Epoch 242
Validation auc = 0.723474
Epoch 243
Validation auc = 0.721596
Epoch 244
Loss = 1.0636e-02, PNorm = 110.4611, GNorm = 0.1512, lr_0 = 6.6625e-04
Validation auc = 0.723474
Epoch 245
Validation auc = 0.722535
Epoch 246
Validation auc = 0.722066
Epoch 247
Validation auc = 0.720657
Epoch 248
Validation auc = 0.719249
Epoch 249
Loss = 5.3924e-04, PNorm = 110.5488, GNorm = 0.0428, lr_0 = 6.6056e-04
Validation auc = 0.719249
Epoch 250
Validation auc = 0.722535
Epoch 251
Validation auc = 0.725352
Epoch 252
Validation auc = 0.723944
Epoch 253
Validation auc = 0.723005
Epoch 254
Loss = 3.2747e-04, PNorm = 110.6079, GNorm = 0.0549, lr_0 = 6.5491e-04
Validation auc = 0.724413
Epoch 255
Validation auc = 0.721596
Epoch 256
Validation auc = 0.720188
Epoch 257
Validation auc = 0.717371
Epoch 258
Validation auc = 0.712207
Epoch 259
Loss = 2.5622e-03, PNorm = 110.6573, GNorm = 0.2161, lr_0 = 6.4931e-04
Validation auc = 0.708920
Epoch 260
Validation auc = 0.707042
Epoch 261
Validation auc = 0.697183
Epoch 262
Validation auc = 0.695775
Epoch 263
Validation auc = 0.694836
Epoch 264
Loss = 5.2175e-02, PNorm = 110.7303, GNorm = 0.1612, lr_0 = 6.4376e-04
Validation auc = 0.692488
Epoch 265
Validation auc = 0.692488
Epoch 266
Validation auc = 0.698122
Epoch 267
Validation auc = 0.713146
Epoch 268
Validation auc = 0.717840
Epoch 269
Loss = 4.0220e-03, PNorm = 110.8471, GNorm = 0.8687, lr_0 = 6.3826e-04
Validation auc = 0.720188
Epoch 270
Validation auc = 0.723474
Epoch 271
Validation auc = 0.723005
Epoch 272
Validation auc = 0.725352
Epoch 273
Validation auc = 0.728638
Epoch 274
Loss = 4.2275e-03, PNorm = 110.9687, GNorm = 0.3285, lr_0 = 6.3280e-04
Validation auc = 0.732394
Epoch 275
Validation auc = 0.737089
Epoch 276
Validation auc = 0.731925
Epoch 277
Validation auc = 0.730516
Epoch 278
Validation auc = 0.730986
Epoch 279
Loss = 3.5563e-03, PNorm = 111.0823, GNorm = 0.6386, lr_0 = 6.2739e-04
Validation auc = 0.724883
Epoch 280
Validation auc = 0.724883
Epoch 281
Validation auc = 0.718310
Epoch 282
Validation auc = 0.714085
Epoch 283
Validation auc = 0.713615
Epoch 284
Loss = 1.5142e-02, PNorm = 111.1962, GNorm = 0.0809, lr_0 = 6.2203e-04
Validation auc = 0.709859
Epoch 285
Validation auc = 0.709390
Epoch 286
Validation auc = 0.728404
Epoch 287
Validation auc = 0.736150
Epoch 288
Validation auc = 0.735211
Epoch 289
Loss = 9.7638e-03, PNorm = 111.3198, GNorm = 1.4951, lr_0 = 6.1671e-04
Validation auc = 0.737089
Epoch 290
Validation auc = 0.735681
Epoch 291
Validation auc = 0.729577
Epoch 292
Validation auc = 0.722066
Epoch 293
Validation auc = 0.718779
Epoch 294
Loss = 3.9471e-03, PNorm = 111.4388, GNorm = 0.6651, lr_0 = 6.1144e-04
Validation auc = 0.716432
Epoch 295
Validation auc = 0.712676
Epoch 296
Validation auc = 0.711737
Epoch 297
Validation auc = 0.714554
Epoch 298
Validation auc = 0.713146
Epoch 299
Loss = 1.0008e-04, PNorm = 111.5104, GNorm = 0.0081, lr_0 = 6.0621e-04
Validation auc = 0.715493
Model 0 best validation auc = 0.757277 on epoch 99
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.895244
Ensemble test auc = 0.895244
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=2400, out_features=2400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.05, inplace=False)
    (7): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 24,441,601
Moving model to cuda
Epoch 0
Validation auc = 0.683568
Epoch 1
Validation auc = 0.661033
Epoch 2
Validation auc = 0.612207
Epoch 3
Validation auc = 0.673239
Epoch 4
Loss = 6.5117e-01, PNorm = 100.5536, GNorm = 3.6839, lr_0 = 6.5000e-04
Validation auc = 0.578404
Epoch 5
Validation auc = 0.604695
Epoch 6
Validation auc = 0.619718
Epoch 7
Validation auc = 0.608451
Epoch 8
Validation auc = 0.639906
Epoch 9
Loss = 4.4467e-01, PNorm = 100.9266, GNorm = 2.1348, lr_0 = 9.9743e-04
Validation auc = 0.571831
Epoch 10
Validation auc = 0.568545
Epoch 11
Validation auc = 0.563380
Epoch 12
Validation auc = 0.589202
Epoch 13
Validation auc = 0.591549
Epoch 14
Loss = 4.8775e-01, PNorm = 101.2895, GNorm = 10.6258, lr_0 = 9.8890e-04
Validation auc = 0.605164
Epoch 15
Validation auc = 0.623944
Epoch 16
Validation auc = 0.623005
Epoch 17
Validation auc = 0.644131
Epoch 18
Validation auc = 0.604695
Epoch 19
Loss = 3.3555e-01, PNorm = 101.5863, GNorm = 6.0978, lr_0 = 9.8045e-04
Validation auc = 0.682629
Epoch 20
Validation auc = 0.702817
Epoch 21
Validation auc = 0.706573
Epoch 22
Validation auc = 0.703756
Epoch 23
Validation auc = 0.656338
Epoch 24
Loss = 3.6130e-01, PNorm = 101.8723, GNorm = 6.4492, lr_0 = 9.7207e-04
Validation auc = 0.666197
Epoch 25
Validation auc = 0.668545
Epoch 26
Validation auc = 0.634742
Epoch 27
Validation auc = 0.626291
Epoch 28
Validation auc = 0.633803
Epoch 29
Loss = 2.3696e-01, PNorm = 102.1571, GNorm = 1.4365, lr_0 = 9.6376e-04
Validation auc = 0.625822
Epoch 30
Validation auc = 0.642723
Epoch 31
Validation auc = 0.652113
Epoch 32
Validation auc = 0.696244
Epoch 33
Validation auc = 0.702817
Epoch 34
Loss = 1.8521e-01, PNorm = 102.4405, GNorm = 4.7077, lr_0 = 9.5552e-04
Validation auc = 0.695775
Epoch 35
Validation auc = 0.693427
Epoch 36
Validation auc = 0.663380
Epoch 37
Validation auc = 0.642723
Epoch 38
Validation auc = 0.669484
Epoch 39
Loss = 2.7901e-01, PNorm = 102.7768, GNorm = 1.6183, lr_0 = 9.4735e-04
Validation auc = 0.668545
Epoch 40
Validation auc = 0.684507
Epoch 41
Validation auc = 0.690610
Epoch 42
Validation auc = 0.684977
Epoch 43
Validation auc = 0.685446
Epoch 44
Loss = 2.5297e-01, PNorm = 103.1214, GNorm = 6.9684, lr_0 = 9.3925e-04
Validation auc = 0.685915
Epoch 45
Validation auc = 0.679343
Epoch 46
Validation auc = 0.677934
Epoch 47
Validation auc = 0.666197
Epoch 48
Validation auc = 0.661502
Epoch 49
Loss = 2.0961e-01, PNorm = 103.4993, GNorm = 6.9963, lr_0 = 9.3122e-04
Validation auc = 0.678404
Epoch 50
Validation auc = 0.689202
Epoch 51
Validation auc = 0.696714
Epoch 52
Validation auc = 0.695305
Epoch 53
Validation auc = 0.697183
Epoch 54
Loss = 9.0886e-02, PNorm = 103.8771, GNorm = 3.1517, lr_0 = 9.2326e-04
Validation auc = 0.697653
Epoch 55
Validation auc = 0.697183
Epoch 56
Validation auc = 0.698122
Epoch 57
Validation auc = 0.697183
Epoch 58
Validation auc = 0.699531
Epoch 59
Loss = 1.3031e-01, PNorm = 104.2471, GNorm = 4.4638, lr_0 = 9.1537e-04
Validation auc = 0.670423
Epoch 60
Validation auc = 0.678404
Epoch 61
Validation auc = 0.682160
Epoch 62
Validation auc = 0.684038
Epoch 63
Validation auc = 0.689671
Epoch 64
Loss = 1.4106e-01, PNorm = 104.6843, GNorm = 4.3125, lr_0 = 9.0754e-04
Validation auc = 0.684507
Epoch 65
Validation auc = 0.682629
Epoch 66
Validation auc = 0.676526
Epoch 67
Validation auc = 0.678873
Epoch 68
Validation auc = 0.721596
Epoch 69
Loss = 8.6713e-02, PNorm = 105.1095, GNorm = 1.9669, lr_0 = 8.9978e-04
Validation auc = 0.722535
Epoch 70
Validation auc = 0.705634
Epoch 71
Validation auc = 0.690141
Epoch 72
Validation auc = 0.675587
Epoch 73
Validation auc = 0.688263
Epoch 74
Loss = 4.8020e-02, PNorm = 105.4614, GNorm = 2.5873, lr_0 = 8.9209e-04
Validation auc = 0.702817
Epoch 75
Validation auc = 0.710798
Epoch 76
Validation auc = 0.708920
Epoch 77
Validation auc = 0.696244
Epoch 78
Validation auc = 0.687324
Epoch 79
Loss = 9.1077e-02, PNorm = 105.7404, GNorm = 6.9299, lr_0 = 8.8447e-04
Validation auc = 0.680282
Epoch 80
Validation auc = 0.661033
Epoch 81
Validation auc = 0.662441
Epoch 82
Validation auc = 0.659155
Epoch 83
Validation auc = 0.668545
Epoch 84
Loss = 4.7661e-02, PNorm = 106.0423, GNorm = 1.6739, lr_0 = 8.7691e-04
Validation auc = 0.684977
Epoch 85
Validation auc = 0.691080
Epoch 86
Validation auc = 0.697653
Epoch 87
Validation auc = 0.699531
Epoch 88
Validation auc = 0.695305
Epoch 89
Loss = 4.2862e-02, PNorm = 106.3439, GNorm = 1.6760, lr_0 = 8.6941e-04
Validation auc = 0.698122
Epoch 90
Validation auc = 0.696714
Epoch 91
Validation auc = 0.698592
Epoch 92
Validation auc = 0.700469
Epoch 93
Validation auc = 0.698592
Epoch 94
Loss = 3.1503e-02, PNorm = 106.6241, GNorm = 0.7284, lr_0 = 8.6198e-04
Validation auc = 0.698122
Epoch 95
Validation auc = 0.693427
Epoch 96
Validation auc = 0.688263
Epoch 97
Validation auc = 0.674648
Epoch 98
Validation auc = 0.688263
Epoch 99
Loss = 5.7552e-02, PNorm = 106.9058, GNorm = 1.5290, lr_0 = 8.5461e-04
Validation auc = 0.701878
Epoch 100
Validation auc = 0.705164
Epoch 101
Validation auc = 0.687324
Epoch 102
Validation auc = 0.688732
Epoch 103
Validation auc = 0.689202
Epoch 104
Loss = 5.3764e-02, PNorm = 107.1846, GNorm = 3.1135, lr_0 = 8.4730e-04
Validation auc = 0.682160
Epoch 105
Validation auc = 0.694836
Epoch 106
Validation auc = 0.704695
Epoch 107
Validation auc = 0.695305
Epoch 108
Validation auc = 0.696244
Epoch 109
Loss = 5.5932e-03, PNorm = 107.4493, GNorm = 0.9160, lr_0 = 8.4006e-04
Validation auc = 0.700469
Epoch 110
Validation auc = 0.700469
Epoch 111
Validation auc = 0.708920
Epoch 112
Validation auc = 0.711737
Epoch 113
Validation auc = 0.713615
Epoch 114
Loss = 9.6854e-02, PNorm = 107.6976, GNorm = 4.4646, lr_0 = 8.3288e-04
Validation auc = 0.703756
Epoch 115
Validation auc = 0.684507
Epoch 116
Validation auc = 0.675117
Epoch 117
Validation auc = 0.667136
Epoch 118
Validation auc = 0.672770
Epoch 119
Loss = 1.9696e-02, PNorm = 108.0395, GNorm = 2.0660, lr_0 = 8.2576e-04
Validation auc = 0.690141
Epoch 120
Validation auc = 0.708920
Epoch 121
Validation auc = 0.716432
Epoch 122
Validation auc = 0.714554
Epoch 123
Validation auc = 0.707981
Epoch 124
Loss = 4.7923e-02, PNorm = 108.3469, GNorm = 2.5060, lr_0 = 8.1870e-04
Validation auc = 0.706573
Epoch 125
Validation auc = 0.701878
Epoch 126
Validation auc = 0.698122
Epoch 127
Validation auc = 0.700469
Epoch 128
Validation auc = 0.697653
Epoch 129
Loss = 1.2816e-02, PNorm = 108.6279, GNorm = 0.8479, lr_0 = 8.1170e-04
Validation auc = 0.695305
Epoch 130
Validation auc = 0.692958
Epoch 131
Validation auc = 0.685446
Epoch 132
Validation auc = 0.685446
Epoch 133
Validation auc = 0.690141
Epoch 134
Loss = 3.4376e-03, PNorm = 108.8250, GNorm = 0.0569, lr_0 = 8.0476e-04
Validation auc = 0.697183
Epoch 135
Validation auc = 0.702817
Epoch 136
Validation auc = 0.705164
Epoch 137
Validation auc = 0.702817
Epoch 138
Validation auc = 0.700469
Epoch 139
Loss = 4.6096e-02, PNorm = 108.9483, GNorm = 0.2476, lr_0 = 7.9788e-04
Validation auc = 0.717371
Epoch 140
Validation auc = 0.727700
Epoch 141
Validation auc = 0.727700
Epoch 142
Validation auc = 0.714554
Epoch 143
Validation auc = 0.724413
Epoch 144
Loss = 3.4964e-02, PNorm = 109.1573, GNorm = 3.6418, lr_0 = 7.9106e-04
Validation auc = 0.727700
Epoch 145
Validation auc = 0.723005
Epoch 146
Validation auc = 0.717840
Epoch 147
Validation auc = 0.727230
Epoch 148
Validation auc = 0.735211
Epoch 149
Loss = 1.0173e-02, PNorm = 109.3926, GNorm = 0.9329, lr_0 = 7.8430e-04
Validation auc = 0.738498
Epoch 150
Validation auc = 0.744131
Epoch 151
Validation auc = 0.738967
Epoch 152
Validation auc = 0.734742
Epoch 153
Validation auc = 0.729108
Epoch 154
Loss = 4.9196e-03, PNorm = 109.5750, GNorm = 0.1892, lr_0 = 7.7759e-04
Validation auc = 0.718310
Epoch 155
Validation auc = 0.711737
Epoch 156
Validation auc = 0.708920
Epoch 157
Validation auc = 0.712207
Epoch 158
Validation auc = 0.710798
Epoch 159
Loss = 1.2459e-03, PNorm = 109.7139, GNorm = 0.0837, lr_0 = 7.7095e-04
Validation auc = 0.707512
Epoch 160
Validation auc = 0.714554
Epoch 161
Validation auc = 0.728638
Epoch 162
Validation auc = 0.734742
Epoch 163
Validation auc = 0.723474
Epoch 164
Loss = 1.6094e-03, PNorm = 109.8553, GNorm = 0.0345, lr_0 = 7.6436e-04
Validation auc = 0.724413
Epoch 165
Validation auc = 0.727700
Epoch 166
Validation auc = 0.729577
Epoch 167
Validation auc = 0.734742
Epoch 168
Validation auc = 0.750704
Epoch 169
Loss = 3.3260e-02, PNorm = 109.9786, GNorm = 4.9980, lr_0 = 7.5782e-04
Validation auc = 0.752113
Epoch 170
Validation auc = 0.746948
Epoch 171
Validation auc = 0.746009
Epoch 172
Validation auc = 0.744131
Epoch 173
Validation auc = 0.743192
Epoch 174
Loss = 1.8157e-03, PNorm = 110.0986, GNorm = 0.3212, lr_0 = 7.5134e-04
Validation auc = 0.745070
Epoch 175
Validation auc = 0.741315
Epoch 176
Validation auc = 0.736620
Epoch 177
Validation auc = 0.734272
Epoch 178
Validation auc = 0.736150
Epoch 179
Loss = 1.2067e-03, PNorm = 110.1987, GNorm = 0.1340, lr_0 = 7.4492e-04
Validation auc = 0.733333
Epoch 180
Validation auc = 0.730986
Epoch 181
Validation auc = 0.733803
Epoch 182
Validation auc = 0.732864
Epoch 183
Validation auc = 0.732394
Epoch 184
Loss = 9.1017e-04, PNorm = 110.2786, GNorm = 0.0675, lr_0 = 7.3855e-04
Validation auc = 0.731455
Epoch 185
Validation auc = 0.728638
Epoch 186
Validation auc = 0.724883
Epoch 187
Validation auc = 0.722535
Epoch 188
Validation auc = 0.721596
Epoch 189
Loss = 6.2828e-04, PNorm = 110.3214, GNorm = 0.1172, lr_0 = 7.3224e-04
Validation auc = 0.718779
Epoch 190
Validation auc = 0.716432
Epoch 191
Validation auc = 0.715023
Epoch 192
Validation auc = 0.712676
Epoch 193
Validation auc = 0.712676
Epoch 194
Loss = 1.0478e-02, PNorm = 110.3454, GNorm = 0.0920, lr_0 = 7.2598e-04
Validation auc = 0.715962
Epoch 195
Validation auc = 0.715023
Epoch 196
Validation auc = 0.718779
Epoch 197
Validation auc = 0.721127
Epoch 198
Validation auc = 0.716432
Epoch 199
Loss = 5.9660e-04, PNorm = 110.3690, GNorm = 0.1399, lr_0 = 7.1977e-04
Validation auc = 0.712207
Epoch 200
Validation auc = 0.707981
Epoch 201
Validation auc = 0.721127
Epoch 202
Validation auc = 0.735211
Epoch 203
Validation auc = 0.739906
Epoch 204
Loss = 2.8683e-02, PNorm = 110.4523, GNorm = 3.8101, lr_0 = 7.1362e-04
Validation auc = 0.739437
Epoch 205
Validation auc = 0.738967
Epoch 206
Validation auc = 0.741315
Epoch 207
Validation auc = 0.738028
Epoch 208
Validation auc = 0.732394
Epoch 209
Loss = 9.5821e-04, PNorm = 110.5825, GNorm = 0.0679, lr_0 = 7.0752e-04
Validation auc = 0.725822
Epoch 210
Validation auc = 0.730516
Epoch 211
Validation auc = 0.746009
Epoch 212
Validation auc = 0.749765
Epoch 213
Validation auc = 0.739906
Epoch 214
Loss = 6.6392e-02, PNorm = 110.6829, GNorm = 3.3424, lr_0 = 7.0147e-04
Validation auc = 0.734272
Epoch 215
Validation auc = 0.733333
Epoch 216
Validation auc = 0.733803
Epoch 217
Validation auc = 0.735681
Epoch 218
Validation auc = 0.730516
Epoch 219
Loss = 2.1506e-03, PNorm = 110.8547, GNorm = 0.3369, lr_0 = 6.9548e-04
Validation auc = 0.727700
Epoch 220
Validation auc = 0.726291
Epoch 221
Validation auc = 0.726291
Epoch 222
Validation auc = 0.723474
Epoch 223
Validation auc = 0.719718
Epoch 224
Loss = 3.8613e-03, PNorm = 110.9933, GNorm = 0.3028, lr_0 = 6.8953e-04
Validation auc = 0.707512
Epoch 225
Validation auc = 0.698122
Epoch 226
Validation auc = 0.692488
Epoch 227
Validation auc = 0.692019
Epoch 228
Validation auc = 0.687324
Epoch 229
Loss = 8.7102e-04, PNorm = 111.0888, GNorm = 0.0428, lr_0 = 6.8364e-04
Validation auc = 0.684977
Epoch 230
Validation auc = 0.684038
Epoch 231
Validation auc = 0.683568
Epoch 232
Validation auc = 0.683568
Epoch 233
Validation auc = 0.684038
Epoch 234
Loss = 5.1152e-04, PNorm = 111.1347, GNorm = 0.0497, lr_0 = 6.7779e-04
Validation auc = 0.684507
Epoch 235
Validation auc = 0.685446
Epoch 236
Validation auc = 0.697653
Epoch 237
Validation auc = 0.701408
Epoch 238
Validation auc = 0.701878
Epoch 239
Loss = 3.4657e-04, PNorm = 111.1597, GNorm = 0.0236, lr_0 = 6.7200e-04
Validation auc = 0.703286
Epoch 240
Validation auc = 0.703756
Epoch 241
Validation auc = 0.705634
Epoch 242
Validation auc = 0.708920
Epoch 243
Validation auc = 0.708920
Epoch 244
Loss = 5.3500e-04, PNorm = 111.1753, GNorm = 0.0137, lr_0 = 6.6625e-04
Validation auc = 0.707981
Epoch 245
Validation auc = 0.707512
Epoch 246
Validation auc = 0.707042
Epoch 247
Validation auc = 0.706103
Epoch 248
Validation auc = 0.704695
Epoch 249
Loss = 3.3373e-04, PNorm = 111.1879, GNorm = 0.0204, lr_0 = 6.6056e-04
Validation auc = 0.704225
Epoch 250
Validation auc = 0.703286
Epoch 251
Validation auc = 0.703286
Epoch 252
Validation auc = 0.705164
Epoch 253
Validation auc = 0.705634
Epoch 254
Loss = 1.4105e-04, PNorm = 111.1998, GNorm = 0.0058, lr_0 = 6.5491e-04
Validation auc = 0.706573
Epoch 255
Validation auc = 0.706573
Epoch 256
Validation auc = 0.707042
Epoch 257
Validation auc = 0.707512
Epoch 258
Validation auc = 0.707512
Epoch 259
Loss = 1.6388e-04, PNorm = 111.2084, GNorm = 0.0247, lr_0 = 6.4931e-04
Validation auc = 0.707981
Epoch 260
Validation auc = 0.708451
Epoch 261
Validation auc = 0.708920
Epoch 262
Validation auc = 0.708451
Epoch 263
Validation auc = 0.708920
Epoch 264
Loss = 1.3609e-04, PNorm = 111.2150, GNorm = 0.0026, lr_0 = 6.4376e-04
Validation auc = 0.708920
Epoch 265
Validation auc = 0.707512
Epoch 266
Validation auc = 0.709390
Epoch 267
Validation auc = 0.711737
Epoch 268
Validation auc = 0.710798
Epoch 269
Loss = 2.0503e-04, PNorm = 111.2232, GNorm = 0.0161, lr_0 = 6.3826e-04
Validation auc = 0.711268
Epoch 270
Validation auc = 0.713146
Epoch 271
Validation auc = 0.712207
Epoch 272
Validation auc = 0.711737
Epoch 273
Validation auc = 0.712207
Epoch 274
Loss = 1.8304e-04, PNorm = 111.2322, GNorm = 0.0195, lr_0 = 6.3280e-04
Validation auc = 0.711268
Epoch 275
Validation auc = 0.710798
Epoch 276
Validation auc = 0.710798
Epoch 277
Validation auc = 0.709859
Epoch 278
Validation auc = 0.708920
Epoch 279
Loss = 1.1963e-04, PNorm = 111.2403, GNorm = 0.0093, lr_0 = 6.2739e-04
Validation auc = 0.709390
Epoch 280
Validation auc = 0.708920
Epoch 281
Validation auc = 0.708451
Epoch 282
Validation auc = 0.707512
Epoch 283
Validation auc = 0.706573
Epoch 284
Loss = 3.2724e-04, PNorm = 111.2459, GNorm = 0.0168, lr_0 = 6.2203e-04
Validation auc = 0.705164
Epoch 285
Validation auc = 0.705164
Epoch 286
Validation auc = 0.705164
Epoch 287
Validation auc = 0.705164
Epoch 288
Validation auc = 0.705164
Epoch 289
Loss = 7.0003e-05, PNorm = 111.2520, GNorm = 0.0036, lr_0 = 6.1671e-04
Validation auc = 0.704695
Epoch 290
Validation auc = 0.704695
Epoch 291
Validation auc = 0.704695
Epoch 292
Validation auc = 0.704695
Epoch 293
Validation auc = 0.704695
Epoch 294
Loss = 2.0721e-04, PNorm = 111.2566, GNorm = 0.0029, lr_0 = 6.1144e-04
Validation auc = 0.704225
Epoch 295
Validation auc = 0.703756
Epoch 296
Validation auc = 0.703756
Epoch 297
Validation auc = 0.703756
Epoch 298
Validation auc = 0.703756
Epoch 299
Loss = 6.0708e-05, PNorm = 111.2605, GNorm = 0.0054, lr_0 = 6.0621e-04
Validation auc = 0.703756
Model 0 best validation auc = 0.752113 on epoch 169
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.907285
Ensemble test auc = 0.907285
10-fold cross validation
	Seed 0 ==> test auc = 0.886213
	Seed 1 ==> test auc = 0.880795
	Seed 2 ==> test auc = 0.860325
	Seed 3 ==> test auc = 0.907285
	Seed 4 ==> test auc = 0.866948
	Seed 5 ==> test auc = 0.862131
	Seed 6 ==> test auc = 0.881999
	Seed 7 ==> test auc = 0.888621
	Seed 8 ==> test auc = 0.895244
	Seed 9 ==> test auc = 0.907285
Overall test auc = 0.883685 +/- 0.016062
Elapsed time = 0:12:46
